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ABSTRACT 



The Program for Area Concentration Achievement Testing 
(PACAT) produces the cooperative assessment instrument known as the Area 
Concentration Achievement Test (ACAT) . The ACAT uses a model designed 
specifically to measure curricular strengths and weaknesses and to provide 
this information at the departmental level. PACAT has developed 57 
descriptive curriculum patterns for 12 disciplines based on survey 
information from 5,699 academic departments. The number of ACATs administered 
annually has grown from 400 in 1988 to over 2,600 in 1992. The PACAT was 
developed with support from the Fund for the Improvement of Post secondary 
Education (FIPSE) , and this report is the final report under the FIPSE grant. 
During the period of FIPSE support, PACAT staff published 3 articles and made 
23 presentations concerning the project. PACAT staff also conducted workshops 
or discussion groups and assisted in the conduct of two doctoral 
dissertations and one master's thesis. Approximately 9,000 sets of survey 
results and curricula have been distributed by PACAT. Test items for the ACAT 
are developed by working with faculty in participating departments to 
construct items that reflect their expectations of performance at a level 
appropriate for graduating seniors. Area concentration achievement tests are 
available in the following fields: agriculture, art, biology, communication, 
literature in English, political science, psychology, and social work. It is 
a measure of PACAT 's success that it continues beyond the FIPSE support 
period, working with over 100 academic departments in 28 states. The body of 
the report contains: (1) "Project Overview"; (2) "Purpose"; (3) "Background 

and Origins"; (4) "Project Descriptions"; (5) "Project Results"; and (6) 
"Summary and Conclusions." Nine appendixes present supplemental information, 
including content area survey descriptions, lists of participants and ACAT 
users, and sample materials. (SLD) 



******************************************************************************** 

♦ Reproductions supplied by EDRS are the best that can be made ♦ 

♦ from the original document . ♦ 

******************************************************************************** 




TM 028 064 



Program for Area Concentration Achievement Testing 



VO 



s 



Grantee Organization: 

Austin Peay State University 
College Street 
Clarksville, TN 37044 



Grant Number: P 1 1 4? & S 1 1 1 



Project Dates: 

Starting Date: August 1, 1988 
Ending Date: July 31, 1992 

Number of Months: 48 (36 + 12 month no cost extension) 



Project Director: 

Anthony Golden 
Department of Psychology 
Austin Peay State University 
Clarksville, TN 37044 
Telephone: (615) 648-7451 



FIPSE Program Officer(s): 

Constance Cook 
Lewis Greenstein 



Grant Award: 



Year 1 $83,085.00 

Year 2 $85,654.00 

Year 3 $88,400.00 

Year 4 (no cost extension) 
Total $257,139.00 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
J CENTER (ERIC) 

(fTThis document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




2 



BEST COPY AVAILABLE 



Table of Contents 



Summary • • • • 
Executive Summary 



Project Overview 2 

Purpose 2 

Background and Origins 2 

Project Descriptions 3 

Project Results 3 

Summary and Conclusions 4 

Body of Report 5 

Project Overview 5 

Purpose 6 

Background and Origins 7 

Project Descriptions 8 

Project Results 9 

Summary and Conclusions 17 

Appendices 

I: Information for FIPSE 19 

II: PACAT Staff 1988-92 21 

III: Departments Using the ACAT 23 

IV: Content Area Surveys and Curriculum Patterns- • • • 27 

V: Publications, Presentations and Workshops 59 

VI: Results of the Assessment Planning Survey 63 

VII: Sample Materials Developed by PACAT 65 

VIII: Sample Score Report 84 

IX: Reports of Consultants and Other Comments • • • • 93 




3 



Program for Area Concentration Achievement Testing 

Summary 



PACAT produces the cooperative assessment instrument known as the Area Concentration 
Achievement Test (ACAT). The AC AT uses a model designed specifically to measure 
curriculular strengths and weaknesses and to provide this information at the departmental 
level. PACAT has developed 57 descriptive curriculum patterns for 12 disciplines based 
upon survey information from 5,699 academic departments. Over 9,000 sets of information 
concerning the findings have been distributed by mail and in person. The number of ACATs 
administered annually has gone from 400 in 1988 to over 2,600 in 1992. PACAT is 
continuing to work with over 100 academic departments in 28 states. 
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Program for Area Concentration Achievement Testing 
Executive Summary 



Project Overview: 

The Project for Area Concentration Achievement Testing (PACAT) was developed 
during 1988-92 with support from the Fund for the Improvement of Post Secondary 
Education. The methodologies used by PACAT were developed by Dr. Anthony Golden, 
Director of PACAT, to address the ne^ for direct faculty involvement in a system for the 
evaluation of disciplinary curricula through standardized instruments. PACAT is responsible 
for the production and management of the faculty consortium derived measurement 
instruments known collectively as the Area Concentration Achievement Tests (ACAT). 

Beginning with a consortium of 5 Psychology Departments in 1983, PACAT now 
serves 108 departments in 8 disciplines across a 28 state area. FIPSE funds have permitted 
PACAT to add instruments in 5 disciplines, with a sixth ready for distribution, and to 
complete the preliminary work necessary to add 2 more. Over 7,000 ACATs have been 
administered. FIPSE funds also were used to conduct surveys of 12,500 academic 
departments concerning their curricular emphases. The 5,699 returned surveys have been 
used to construct 57 descriptive curricula for 12 academic disciplines. Approximately 9,000 
sets of survey results and curricula have been distributed nationally to date. 

During the period of FIPSE support, PACAT staff published 3 articles and made 23 
presentations concerning the project. In addition, PACAT staff conducted 7 workshops 
and/or discussion groups at various sites. PACAT assisted in the conduct of 2 doctoral 
dissertations and 1 master's thesis and was featured in two additional publications which dealt 
with undergraduate teaching. 

It is a measure of PACAT’s success that it continues beyond the FIPSE support 
period and is still exhibiting strong growth. 

Purpose: 

According to the Bulletin of the American Association for Higher Education, by 1987 
67% of the states had or were considering assessment mandates or initiatives. A rnore recent 
survey conducted by ACE and Winthrop University suggests that by 1990, approximately 
89% of institutions were either planning or implementing comprehensive assessment 
programs primarily using local measures (39%) and performance based measures (38%). 

The most frequently cited reasons for initiating outcomes assessment were accrediting board 
standards (46%) and state education policies (39%). 

It has been asserted that the outcomes assessment movement should address individual 
student learning and growth across the academic career using "authentic" methods of 
assessment that most closely resemble what a student normally does in the classroom. While 
part of the outcomes assessment movement will undoubtedly always be the measurement of 
individual student learning, the emphasis on the evaluation of institutional priorities and 
curricula makes it essential that specialized measurement models also be available for this 
purpose. 

A curriculum assessment instrument must reflect the goals of the department, 
including different concentrations within a major. Since originally formulating a response to 
these needs, the Project for Area Concentration Achievement Testing (PACAT) has obtained 
assessment planning information from 1,290 academic departments. Although it was 
originally anticipate that a number of departments would elect to use standardized 
instruments, the percentage has turned out to be considerably larger than expected (55%). 

Background and Origins: 

In 1983, Dr. Anthony Golden developed a model for major field assessment for a 
consortium of Psychology departments. By 1988, when PACAT began receiving support 
from FIPSE, the approach had been extended to consortia in Social Work and Political 
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Science and the project was providing instruments to academic departments in Tennessee, 
Kentucky, and Georgia. The model gained widespread accept^ce as a result of its ability to 
meet differing departmental needs; to provide curriculum specific feedback and assistance 
with interpretation, and because of its cost-effectiveness. 

Austin Peay State University is the smallest of the Tennessee Board of Regents 4 year 
institutions. As a result, it had few resources to contribute toward expanding the project. 
Furthermore, the largely extramural nature of PACAT's work made it essential that it 
actively seek external sources of funding while increasing its own support through user fees. 
The bulk of the funds for national expansion of the project came from FIPSE, with Austin 
Peay State University contributing additional reassigned time for Dr. Golden as well as office 
space, long distance telephone service, and over $14,000 in computer and office equipment. 

Project Description: 

PACAT is responsible for producing the cooperative assessment instrument known as 
the Area Concentration Achievement Test (ACAT). The ACAT is unique in that it uses a 
model designed specifically to measure curriculular strengths and weaknesses and to provide 
this information to faculty at the departmental level. 

Through a series of national surveys, PACAT has obtained information concerning 
curricular structure and content area requirements from academic departments in 15 different 
disciplines. The information has been used to match departments with similar content 
requirements for the purpose of cooperative assessment using the ACAT. At the present 
time, this represents one of the largest data bases available concerning what is actually being 
taught and required at the individual departmental level. 

Test items for the ACAT are obtained by working with faculty in participating 
departments to construct items which reflect their expectations of .performance at a level 
appropriate for graduating seniors. In this way, faculty have a direct involvement in 
determining the content of the instrument. Information generated using this approach has 
greater credibility with the faculty using the ACAT. 

The ACAT is uniquely structured to be adapted to a variety of departmental 
requirements. It provides many of the benefits of both nationally normed and locally 
developed instruments, producing information specifically applicable to the individual 
department within a larger national context of departments with similar curricula. The use of 
this type of assessment instrument is beginning to have a significant positive impact on 
campuses across the nation. 

Project Results: 

PACAT primarily is responsible for developing materials which can act as an agent of 
change on other campuses. As such, it is often difficult to determine the precise irnpact of 
the project with respect to change. We can, however, estimate its impact by examining the 
ways in which the ACAT is being used. 

The ACAT in social work currently is being used by a number of departments to 
comply with the Council for Social Work Education's (CSWE) "Baccalaureate Interpretive 
Guidelines Evaluative Standard 2-Outcomes" for program affirmation and reaffirmation. The 
College of Social Work at Ohio State University currently is using the ACAT for 
reaffirmation and on an experimental basis to determine the "comparability of the preparation 
in foundation content for ... newly initiated extended campus programs with [the] main 
campus program in Columbus". 

The ACATs in Psychology and Social Work have been or are being used at Regis 
University, Austin Peay State University, Pikeville College, and Jamestown College to 
evaluate entry level knowledge and the degree to which an introductory level course 
contributes to the development of the knowledge base in the discipline. Fourteen institutions 
currently are using the ACAT as part of a pre- post-test plan for ev^uating the impact of the 
major on development of the disciplinary Imowledge base. The University of Alabama has 
used the ACAT in psychology to assess the efficacy of a special course intended to prepare 



PACAT - Grant ii'Pllb BSlTll - Final Report - 3 

O 

ERIC 



6 



graduate students to teach an introductory psychology course. Finally, ACATs have served 
as the basis for two dissertations and a masters thesis. 

PACAT has developed 57 descriptive content area requirement patterns for 12 
disciplines based upon information from 5,699 academic departments nationally. Over 9,000 
sets of information concerning the survey findings and curricula have been distributed by 
mail and by PACAT staff at various meetings and workshops. 

Of the six new instruments which were to have been developed and fielded, six were 
developed but only five were fielded during the period of FIPSE funding. Revisions of the 
instruments to conform to the national surveys was accomplished to provide all new versions 
requested by participating departments. 

The number of ACATs administered annually has gone from 400 in the first year of 
FIPSE funding to over 2,600 in 1992. Project revenues have also grown from zero in 1988 
to over $17,000 in 1992. Although it has exhibited excellent growth since the receipt of 
FIPSE support in 1988, PACAT has not expanded at the rate anticipated at that time. The 
reason appears to be that the planning and implementation of outcomes assessment at the 
departmental level has lagged behind state mandates and institutional decision making. 

In addition to the evaluation of the project's success against the stated goals and ‘ 
objectives, other indications of the success of the project can be found in its ability to 
disseminate its findings to the academic community nationally. PACAT staff have published 
3 articles and made 23 presentations, conducting 6 workshops and/or discussion groups 
during the period of FIPSE funding. 

PACAT was formally evaluated by Drs. Reid Johnson, Gary Pike, and Jason 
Millman. Drs. Johnson and Pike pronounced the project to be in excellent condition and 
making a major contribution to higher education curricular reform. Dr. Millman of Cornell 
University reviewed the statistical procedures used by the project and found that, although far 
from psychometrically pure, they were innovative and appropriate to the task of outcomes 
assessment and curricular examination and worth pursuing. 

Summary and Conclusions: 

Overall, PACAT achieved its primary goal which was to make available nationally 
outcomes assessment instruments for the major which were generated by participating faculty 
and reflected individual departmental curricular emphases. The success of PACAT has been 
such that it is continuing to provide services to over 100 institutions beyond the termination 
of FIPSE support. Evaluations of the project against the goals and objectives set forth in the 
original application to FIPSE indicate that PACAT was ame to meet or exceed nearly all of 
them. The only exceptions are where it has become evident that national growth in outcomes 
assessment is proceeding at a slower than anticipated pace, impacting negatively on 
PACAT's growth. 

Members of the PACAT staff have been actively representing the project at 
assessment meetings, disciplinary meetings, and on individual campuses. Given the small 
size of PACAT's staff, an excellent job has been done to disseminate information about the 
project to the widest possible audience. 

External reviewers of the project, both formal and informal, have been exceedingly 
positive. The general impression is that PACAT has made a positive impact on outcomes 
assessment and that the information provided through the content area surveys, descriptive 
curricula and test results have provided essential information to faculty making decisions at 
the departmental level. 

PACAT is continuing to provide its services. At this time, plans are being made to 
open new consortia in Sociology and Industrial Engineering. The new ACAT in History will 
be fielded in December 1992 and should be in full circulation by October 1993. We are 
continuing to work with the consortium that is being organized in Criminal Justice. As we 
have done for the past 4 years, we will continue to provide information to departments 
concerning outcomes assessment, assessment in the major, our national surveys, and other 
national and institutional assessment projects that might serve their needs. Because the 
evaluation of the project is built in to its routine operations, it will continue with the project. 
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Program for Area Concentration Achievement Testing 

Final Report 



Project Overview: 

The Project for Area Concentration Achievement Testing (PACAT) was developed 
during 1988-92 with support from the Fund for the Improvement of Post Secondary 
Education. The methodologies used by PACAT were developed by Dr. Anthony Golden, 
Director of PACAT, to address the need for faculty vestment in a system for the evaluation 
of curricula through standardized instruments. PACAT is responsible for the production and 
management of the faculty consortium derived instruments known as the Area Concentration 
Achievement Tests (ACAT). 

Beginning with a consortium of 5 Psychology Departments in 1983 and extending to 
additional consortia in Social Work and Political Science by 1987, PACAT now serves 108 
departments in 8 disciplines across a 28 state area (see Appendix III). FIPSE funds have 
permitted PACAT to add instruments in 5 disciplines, with a sixth ready for distribution, and 
to complete the preliminary work necessary to add 2 more. Over 7,000 ACATs have been 
administered. 

FIPSE funds also were used to conduct surveys of 12,500 academic departments 
concerning their curricular emphases. The 5,699 returned surveys have been used to 
construct 57 descriptive curricula for 12 academic disciplines (see Appendix IV). 
Approximately 9,000 sets of survey results and curricula have been distributed nationally to 
date. 

During the period of FIPSE support, PACAT staff published 3 articles and made 23 
presentations concerning the project. In addition, PACAT staff conducted 7 workshops 
and/or discussion groups at various sites. PACAT assisted in the conduct of 2 doctoral 
dissertations and 1 master's thesis and was featured in two additional publications which dealt 
with undergraduate teaching, (see Appendix V) 
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It is a measure of PACAT's success that it continues beyond the FIPSE support 
period and is still exhibiting strong growth. 

Purpose: 

In 1979, the American Association of State Colleges and Universities (AASCU), 
supported by the Fund for the Improvement of Post Secondary Education (FIPSE), began the 
Academic Program Evaluation Project. In the same year, the Tennessee Higher Education 
Commission introduced a performance funding formula to encourage the evaluation and 
improvement of academic programs. A 1984 report from the National Institute of Education 
supported the use of assessment to determine whether or not educational goals were being 
met. In 1986, the AASCU Conference on Legislative Action and Assessment summarized 
APEP and introduced additional models, including the ACAT, developed at non-APEP 
institutions. The same year, the American Association for Higher Education, with support 
from FIPSE, held the first Conference on Assessment in Higher Education. 

According to the Bulletin of the American Association for Higher Education^, by 
1987 67% of the states had or were considering assessment mandates or initiatives. A more 
recent survey conducted by ACE and Winthrop University^ suggests that by 1990, 
approximately 89% of institutions were either planning or implementing comprehensive 
assessment programs primarily using local measures (39%) and performance based measures 
(38%). The most frequently cited reasons for initiating outcomes assessment were 
accrediting board standards (46%) and state education policies (39%). 

It has been asserted that the outcomes assessment movement should address individual 
student learning and growth across the academic career using "authentic" methods of 

'1 

assessment that most closely resemble what a student normally does in the classroom. 

1. March, 1987. 

2. A preliminary report appears in The Exchange, South Carolina Higher Education Assessment Network, Winter 
1990. The results also were presented as part of Larson, R. (moderator), Johnson, R.(*), Prus, J.(*), Nichols, J., 
and Wolff, L. Who’s Doing What and Why? Results of Two National Assessment Surveys. Sixth AAHE 
Conference on Assessment in Higher Education, San Francisco, June 1991. 

3. Cross, K. (moderator), Ewell, P., Mentkowski, M., and Moran, E. Conceptual Framework for Assessment - 
Catching Theory up With Practice. Sixth AAHE Conference on Assessment in Higher Education, San Francisco, 
June 1991. 
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While part of the outcomes assessment movement will undoubtedly always be the 
measurement of individual student learning, the emphasis on the evaluation of institutional 
priorities and curricula makes it essential that specialized measurement models also be 
available for this purpose. 

A curriculum assessment instrument must reflect the goals of the department, 
including different concentrations within a major.'* Standardized tests with fixed content 
yield little usable information in this context. Departments often are unable to make 
meaningful comparisons of their performance to that of departments with similar curricula or 
to their own performance in previous years. Norm-referenced scores may indicate a 
department's relative standing but can be misleading with respect to its actual level of 
performance. To provide a more complete and accurate picture of departmental 
performance, a criterion-referenced, percent-correct score is also needed. 

Since originally formulating a response to these needs, the Project for Area 
Concentration Achievement Testing (PACAT) has obtained assessment planning information 
from 1,290 academic departments (see Appendix VI). Although it was originally anticipated 
that a number of departments would elect to use standardized instruments, the percentage has 
turned out to be considerably larger than expected. On the other hand, our experience also 
suggests that at least 2 years and sometimes 3 or more years elapse between the time that a 
department begins its dialogue concerning assessment and the time that it begins to 
implement a plan. 

Background and Origins: 

In 1983, Dr. Anthony Golden developed a model for major field assessment for a 
consortium of Psychology departments. By 1988, when PACAT began receiving support 
from FIPSE, the approach had been extended to consortia in Social Work and Political 



4. Golden, A. (moderator), Smith, M., Cone, A., and Kidda, M. The Area Concentration Achievement Test 
(ACAT): Different Strategies for Assessing the Psychology Major. Sixth AAHE Conference on Assessment in 
Higher Education, San Francisco, June 1991. 
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Science and the project was providing instruments to academic departments in Tennessee, 
Kentucky, and Georgia. The model, based on the principle that academic assessment should 
originate from within the academic community, gained widespread acceptance as a result of 
its ability to meet differing departmental needs; to provide curriculum specific feedback and 
assistance with interpretation, and because of its cost-effectiveness. 

PACAT originally was organized as one of several functions of the Behavioral 
Sciences Research Group at Austin Peay State University, organized directly under the 
Vice-President for Academic Affairs. Dr. Golden was provided with 1/12 reassigned time 
(equivalent to one 3 quarter hour course per year), a 1/2 time graduate assistant, mainframe 
computer access, and some copying costs. However, the increasing volume of telephone 
calls and written requests for information and assistance with new instruments made it 
evident that significant additional support was needed. 

Austin Peay State University is the smallest of the Tennessee Board of Regents 4 year 
institutions. As a result, it had few resources to contribute toward expanding the project. 
Furthermore, the largely extramural nature of PACAT's work made it essential that it 
actively seek external sources of funding while increasing its own support through user fees. 
The bulk of the funds for national expansion of the project came from FIPSE, with Austin 
Peay State University contributing additional reassigned time for Dr. Golden as well as 
providing 200 square feet of new office space, enhanced telephone service, and over $14,000 
in computer and office equipment. 

Project Description: 

PACAT is a national project, created largely through FIPSE support, responsible for 
producing the cooperative assessment instrument known as the Area Concentration 
Achievement Test (AC AT). The ACAT is unique in that it uses a model designed 
specifically to measure curriculular strengths and weaknesses and to provide this information 
to faculty at the departmental level. 
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Through a series of national surveys, PACAT has obtained information concerning 
curricular structure and requirements from academic departments in the fields of art, biology, 
chemistry, communications, criminal justice, English literature, geology, history, 
mathematics, physics, political science, psychology, public management, social work, and 
sociology (see Appendix IV). The information has been used to match departments with 
similar content requirements for the purpose of cooperative assessment using the ACAT. At 
the present time, this represents one of the largest data bases available concerning what is 
actually being taught and required at the individual departmental level. 

Test items for the ACAT are obtained by working with faculty in participating 
departments to construct items which reflect their expectations of performance at a level 
appropriate for graduating seniors. In this way, faculty have a direct continuing vestment in 
the content of the instrument. Test scores generated using this approach have greater 
credibility with the faculty in the departments using the ACAT and are therefore more likely 
to lead to meaningful reforms. 

The ACAT is intended primarily as a tool by which to measure curricular structure 
and integrity, although it can be used to assess individual student performance. The test is 
uniquely structured to be adapted to a variety of departmental emphases and requirements. 
The ACAT provides many of the benefits of both nationally normed and locally developed 
instruments, producing information specifically applicable to the individual department within 
a larger national context of departments with similar curricula. The use of this type of 
assessment instrument is beginning to have a significant positive impact on campuses across 
the nation. 

Project Results: 

PACAT primarily is responsible for developing materials which can act as an agent of 
change on other campuses. As such, it often is difficult to determine the precise impact of 
the project with respect to change. We can, however, estimate its impact by examining the 
ways in which the ACAT is being used. 
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The ACAT in social work currently is being used by a number of departments to 
comply with the Council for Social Work Education's (CSWE) "Baccalaureate Interpretive 
Guidelines Evaluative Standard 2-Outcomes" for program affirmation and reaffirmation.^ 

The College of Social Work at Ohio State University currently is using the ACAT for 
reaffirmation and on an experimental basis to determine the "comparability of the preparation 
in foundation content for ... newly initiated extended campus programs with [the] main 
campus program in Columbus"^. 

The ACATs in Psychology and Social Work have been or are being used at Regis 
University, Austin Peay State University, Pikeville College, and Jamestown College to 
evaluate entry level knowledge and the degree to which an introductory level course 
contributes to the development of the knowledge base in the discipline. Fourteen institutions 
currently are using the ACAT as part of a pre- post-test plan for evaluating the impact of the 
major on development of the disciplinary knowledge base. The University of Alabama has 
used the ACAT in psychology to assess the efficacy of a special course intended to prepare 
graduate students to teach an introductory psychology course. Finally, ACATs have served 
as the basis for two dissertations and a masters thesis. 

The success of this project must also be evaluated against its original goals and the 
objectives by which they were to be achieved (see original application). 

Goal 1: To determine the appropriate core courses and content area emphases for 24 

separate academic disciplines through the use of a two stage multi-state survey. The 
first stage will provide information concerning the content areas required for the 
major at responding institutions. The second stage will ascertain relative emphases of 
the content areas and will statistically determine the smallest number of common 
patterns of content areas which best represent the largest number of departments. 



5. Golden, A., Carter, G., and Ferraro, E. The ACAT: A Model for Coiyiortiuin Based Outcomes Assessment in 
Social Work. Council for Social Work Education Annual Program Meeting, Kansas City, MO, 1992. 

6. Toomey, personal communication, 1992 
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Information concerning both stages of the process will be reported back to 

participating departments. 

Objective 1. To achieve a 25% response rate (minimum of 500 departments per 
year) on the Phase 1 surveys mailed to 2,000 departments representing 
eight different academic disciplines at public and private institutions in a 
14-state region by the end of each project year. 

Outcome. An overall response rate of 45.6% (5,699 departments) was achieved 
for the 12,500 surveys mailed to departments in a total of 15 disciplines. 
Although the original objective called for surveys to be sent to 
departments in 24 disciplines, the surveys were extended to 50 states, the 
District of Columbia, and the territories. Both the number of surveys sent 
and the number received exceed the objective. 

Objective 2. To achieve a 65% response rate (325 departments per year) on the 
Phase 2 survey of at least 500 Phase 1 respondents by the end of each 
project year. 

Outcome. During the first year of funding, 367 departments in Psychology and 
Social Work were sent survey results and 101 (27.5%) returned completed 
Phase 2 surveys. This rate was well below that called for by the 
objective. During the second year, 1394 sets of Phase 1 results were sent 
out to participating departments and 495 (35.5%) completed Phase 2 
surveys were returned. Subsequently, the Phase 2 surveys were 
incorporated into the original mailing and were routinely sent back by 
departments with the original curriculum survey portion. An additional 
2,375 sets of results were prepared during the no cost extension period but 
for technical reasons will not be mailed until after November 1, 1992. In 
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order to maximize dissemination, results of the surveys also were sent to 
1,270 institutional assessment coordinators in January 1991. 

Summary. The project met and exceeded this goal during the period of FIPSE 
funding. PACAT has developed 57 descriptive curricular patterns for 12 
disciplines based upon information from 5,699 academic departments nationally. 

Over 9,000 sets of information concerning the survey findings and curricula have 
been distributed with Phase 2 surveys, the assessment coordinator mailing, 
through routine correspondence, and by PACAT staff at various meetings and 
workshops. 

Goal 2: To construct Area Concentration Achievement Tests (ACATs) which conform, for 
each discipline, to the patterns identified through Goal 1 . Goal 2 includes the creation 
and maintenance of pools of test items submitted by faculty members of the 
participating institutions and the use of the item pools for the construction and 
revision of the tests. 

Objective 3. To create test item pools in a minimum of three additional 
disciplines, as a result of submission of test items by participating 
departments (total of 6) by the end of project years 2 and 3. 

Outcome. Test item pools have been created in 6 new disciplines: art; biology; 
history; literature in English; communications; and agriculture. This 
brings the total number of pools to 9 rather than the 6 called for by this 
objective. 

Objectives 4 and 5. To develop ACATs in a minimum of two additional 
disciplines in year 2 and four additional disciplines in year 3. 

Outcome. Five new ACATs were developed and fielded in the areas of 

agriculture, art, communications, literature in English and biology. A 
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sixth ACAT in history was prepared but will not be fielded until 
December 1992. 

Objective 6. To increase the number of items in the 4 extant test item pools (at 
the time of writing, Psychology, Political Science, Geology, and Social 
Work) by an average of 10% annually, as a result of submission of test 
items by participating departments by the end of each project year. 

Outcome. This goal has only partially been met. The following items were 
added to the item pools. 



The item pool for Geology has been retired due to an inability to obtain 
clear assignment of copyright to PACAT. 

Objective 7. To revise the contents of all of the currently available ACATs (at 
the time of writing in 1988, Psychology, Political Science, and Social 
Work) to conform to the information determined in the project surveys by 
the end of project year 2. 

Outcome. This objective has only partially been met. Overall content revisions 
were made in all three instruments. Furthermore, the ACATs in 
Psychology have been revised to conform to 5 of the 9 curricula 
identified. The remaining 4 patterns have not been requested. No 
requests have been made for the ACAT in political science in any format 
other than the original. A similar problem has been encountered with the 
ACAT in Social Work. 



Items Added to the Item Pools 



Political Science 
Psychology 
Social Work 



640 ( + 73 %) 

256 (+ 11 %) 

0 
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Summary. The project partially achieved Goal 2. Of the six new instruments 
which were to have been developed and fielded, six were developed but only five 
were fielded during the period of FIPSE funding. We also were unable to obtain 
any new items in Social Work and obtained fewer than the projected number of 
items in Psychology. Revisions of the instruments to conform to the national 
surveys were accomplished to provide all new versions requested by participating 
departments. 

Goal 3: To provide the examinations constructed in Goal 2 to participating departments and to 
serve as a centralized coordination point for scoring and test production. This goal also 
will provide for the compiling of inter-institutional normative data for test evaluation. 

Objective 8. To obtain participation in PACAT of departments from 3 additional 
states outside of Tennessee by the end of each project year for a total of 9 
new states. 

Outcome. PACAT has exceeded this objective considerably by adding 
departments in 25 new states during the period of FIPSE funding. 

Objective 9. To obtain 75% participation (240 departments in at least 2 new 
disciplines per year) of each year's Phase 2 survey respondents (325) in 
the AC AT testing program by the end of each project year. 

Outcome. Although PACAT has experienced considerable growth (also see 

graph accompanying Objective 10 below), this objective was not met. A 
total of 97 new departments joined PACAT consortia during the period of 
FIPSE funding. A complete list of participating departments is included 
in Appendix III. 
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Departments using the ACAT 



Agriculture (New 1992) 


1 


Art (New 1992) 


1 


Biology 


7 


Literature in English 


7 


Mass Communication 


4 


Political Science 


9 


Psychology 


43 


Social Work 


36 


TOTAL 


108 



Objective 10. To increase the number of ACATs administered annually by 
2,500 by the end of project years 2 and 3. 

Outcome. The number of ACATs administered annually is still below 3,000 
although significant growth continues to take place. 



ACATs Administered Annually 




I • i ^2ZZ2 tooui worit roL mm* — Ui ti Bia 
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Summary. Although it has exhibited excellent growth since the receipt of FIPSE 
support in 1988, PACAT has not expanded at the rate anticipated at that time. 

The reason appears to be based upon a number of factors which were not 
apparent in 1988 and which are not under the control of this project. The 
planning and implementation of outcomes assessment at the departmental level 
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appears to have lagged well behind state mandates and institutional decision 
making. Our experience has been that it often takes 1 to 2 years for a department 
to reach a consensus concerning a comprehensive assessment plan. Since 
PACAT works directly with faculty whenever possible, this has considerably 
slowed our rate of growth. The recent recession has delayed implementation on a 
number of campuses and forced only partial implementation on others. Finally, 
faculty are extremely skeptical of "advertised" assessment methodologies, 
preferring instead to hear from their colleagues about new ideas and techniques. 

Until recently, the professional societies were virtually silent on the issue of 
assessment, effectively eliminating any formal exchange of information on the 
topic among teaching faculty. 

Other Areas of Evaluation: In addition to the formal evaluation of the project's success 
against the stated goals and objectives, other indications of the success of the project can be 
found in its ability to disseminate its findings to the academic community nationally. 
Publications by PACAT staff (see Appendix V): 3 
Presentations by PACAT staff (see Appendix V): 23 
Workshops/Discussion Groups by PACAT staff (see Appendix V): 6 
Other Regional /National Meetings attended by PACAT staff: 3 
Interestingly enough, it has indirectly come to our attention that at least two sessions have been 
held at disciplinary meetings within the past year which concerned the ACAT but which were 
organized by PACAT users not by PACAT staff. The presentations in Social Work and 
Psychology have resulted in a number of telephone calls, all of which have been quite 
favorable. 

PACAT was evaluated by three consultants. Drs. Reid Johnson, Gary Pike, and Jason 
Millman were charged with evaluating PACAT' s contributions to outcomes assessment, the 
steps necessary to help insure continuation of the project beyond the FIPSE funding period, 
and the psychometric soundness of the assessment procedures themselves (see Appendix IX). 
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Drs. Johnson and Pike pronounced the project to be in excellent condition and making a major 
contribution to higher education curricular reform. Dr. Millman of Cornell University 
reviewed the statistical procedures used by the project and found that, although far from 
psychometrically pure, they were innovative and appropriate to the task of outcomes 
assessment and curricular examination. 

In spring of 1992, PACAT requested all participating departments to evaluate the 
services that it provides. Although a dishearteningly small number, the letters which were 
received as well as several unsolicited comments have been included in Appendix IX. The 
letters are very positive and speak well of PACAT' s ability to meet the needs of outcomes 
evaluation. 

Revenues received by the project for its services during the p>eriod 1988-92 are also 
indicative of the success of PACAT and its substantial growth during the period of FIPSE 
support. 

PACAT Annual Revenues 



88-89 




0 10 20 
R«v«nuM tin thousaixto) 



Summary and Conclusions: 

Overall, PACAT achieved its primary goal which was to make available nationally 
outcomes assessment instruments for the major which were generated by participating faculty 
and reflected individual departmental curricular emphases. The success of PACAT has been 



PACAT - Grant ^P116B81711 - Final Report - 17 



such that it is continuing to provide services to over 100 institutions beyond the termination of 
FIPSE support. Evaluations of the project against the goals and objectives set forth in the 
original application to FIPSE indicate that PACAT was able to meet or exceed nearly all of 
them. The only exceptions are where it has become evident that national growth in outcomes 
assessment is proceeding at a slower than anticipated pace and has impacted on PACAT's rate 
of growth as well. 

Members of the PACAT staff have been actively representing the project at assessment 
meetings, disciplinary meetings, and on individual campuses. Given the small size of 
PACAT's staff (see Appendix II), an excellent job has been done to disseminate information 
about the project to the widest possible audience. 

External reviewers of the project, both formal and informal, have been exceedingly 
positive. The general impression is that PACAT has made a positive impact on outcomes 
assessment and that the information provided through the content area surveys, descriptive 
curricula and test results have provided essential information to faculty making decisions at the 
departmental level. 

PACAT is continuing to provide its services. At this time, plans are being made to 
open new consortia in Sociology and Industrial Engineering. The new ACAT in History will 
be fielded in December 1992 and should be in full circulation by October 1993. We are 
continuing to work with the consortium that is being organized in Criminal Justice. As we 
have done for the past 4 years, we will continue to provide information to departments 
concerning outcomes assessment, assessment in the major, our national surveys, and other 
national and institutional assessment projects that might serve their needs. Because the 
evaluation of the project is built in to its routine operations, it will continue with the project. 
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Several lessons have been learned from this project which may assist FIPSE or other 
potential Project Directors. Perhaps the most important is that, for all we perceive outcomes 
assessment as a national movement and a pervasive part of the academic dialogue, our 
perception may be distorted. Although assessment coordinators and many senior 
administrators support outcomes assessment, departmental faculty are still suspicious of the 
process and feel themselves to be largely left out of the formulation of institutional policy with 
regard to assessment. Although there are many notable exceptions to this generalization, it 
would appear that outcomes assessment still must gain additional credibility with the academic 
disciplines before its potential can be fulfilled. 

A related problem derives from the observation that many of the larger research 
institutions are only remotely involved in disciplinary assessment. The officers of professional 
organizations tend to be drawn from the ranks of recognized researchers and scholars. 
Teaching, unfortunately, is often not perceived as either or is seen at best as "soft" research. 
Unless an association has an established emphasis on undergraduate education, it is unlikely to 
perceive outcomes assessment, a largely undergraduate undertaking, as being of particular 
importance. 

Finally, most undergraduate institutions do not have the financial means to 
institutionalize large extramural projects. Fortunately, PACAT is able to generate sufficient 
revenues to offset its cost to Austin Peay State University. For other projects, this might not 
be the case. On the one hand, undergraduate institutions are crucial to the success of 
innovative methods for outcomes assessment. On the other hand, they cannot afford to sustain 
innovation and are inherently distrustful of innovations originating at larger, better funded 
institutions. It would be immensely helpful if FIPSE would lobby the cause of outcomes 
assessment among other potential funding sources. Obviously the prestige which FIPSE' s 
name carries would serve to lend tremendous credibility to outcomes assessment as a fundable 
endeavor. 
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APPENDIX II: PACAT Staff 1988-92 
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PACAT Staff 1988-92 



Director 
Anthony Golden 



Assistant Director 

1988- 89 Marcia Wood-Hart 

1989- 91 Denise Squire 



Secretary 



1988- 89 Marjorie Bigham 

1989- Gayle ShocWey 



Research Assistant 

1988- 89 Patricia LeDuc 

1989- 91 Li-ZungLin 



Graduate Assistant 

1988- 89 Michelle Crain 

1989- 90 Terry Corbin 

1990- 91 Becky Brockel 

1991- 92 Tracy Stecker 
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APPENDIX III: Departments Using the ACAT 
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AGRICULTURE 

Austin Peay State University, Clarksville, TN 



ART 

East Central University, Ada, OK 



BIOLOGY 

East Central University, Ada, OK 

Eastern Oregon State College, LaGrande, OR 

Northern Kentucky University, Highland Heights, KY 

Tougaloo State College, Tougaloo, MS 

University of North Carolina-Charlotte, Charlotte, NC 



ENGLISH LITERATURE 

Austin Peay State University, Clarksville, TN 
Jacksonville State University, Jacksonville, AL 
Johnson C. Smith University, Charlotte, NC 
LaGrange College, LaGrange, GA 
Murray State University, Murray, KY 
Southeastern Oklahoma State University, Durant, OK 
University of Tennessee- Chattanooga, Chattanooga, TN 



MASS COMMUNICATION 

Austin Peay State University, Clarksville, TN 
East Central University, Ada OK 
Johnson C. Smith University, Charlotte, NC 
Northwest Missouri State University, Maryville, MO 



POLITICAL SCIENCE 

Austin Peay State University, Clarksville, TN 
Christopher Newport College, Newport News, VA 
East Central State University, Ada, OK 
Jacksonville State University, Jacksonville, AL 
Middle Tennessee State University, Murfreesboro, TN 
Salisbury State University, Salisbury, MD 
University of Tennessee-Chattanooga, Chattanooga, TN 
University of Tenneessee-Martin, Martin, TN 



PSYCHOLOGY 

Athens State College, Athens, AL 

Austin Peay State University, Clarksville, TN 
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Davidson College, Davidson, NC 

East Central University, Ada OK 

East Tennessee State Univesity, Johnson City, TN 

Fisk University, Nashville, TN 

Jacksonville State University, Jacksonville, AL 

James Madison University, Harrisonburg, VA 

Jamestown College, Jamestown, ND 

Johnson C. Smith University, Charlotte, NC 

LaGrange College, LaGrange, GA 

Loyola University, New Orleans, LA 

Maryville College, Maryville, TN 

Mary Washington College, Fredericksburg, VA 

Memphis State University, Memphis, TN 

Middle Tennessee State University, Murfreesboro, TN 

Morgan State University, Baltimore, MD 

Northeast Louisiana State University, Monroe, LA 

Northern Kentucky State University, Highland Heights, KY 

Northwest Missouri State University, Maryville, MO 

Regis University, Denver, CO 

Russell Sage College, Troy, NY 

Salisbury State University, Salisbury, MD 

Slippery Rock University, Slippery Rock, PA 

St. Leo (Educational Services), St. Leo, FL 

St. Leo (Main Campus), St. L^, FL 

St. Leo (Military Campus), Fort Eustis, VA 

Tennessee State University, Nashville, TN 

Tuscaloosa College, Tuscaloosa, MS 

University of Alaska-Anchorage, Anchorage, AK 

University of Kentucky, Lexington, KY 

University of Michigan-Dearborn, Dearborn, MI 

University of Tennessee-Knoxville, Knoxville, TN 

University of Tennessee-Martin, Martin, TN 

Western Kentucky State University, Bowling Green, KY 

Winthrop University, Rock Hill, SC 



SOCIAL WORK 

Austin Peay State University, Clarksville, TN 

Belmont University, Nashville, TN 

Bethany College, Lindsborg, KS 

Castleton State College, Castleton, VT 

Central Missouri State University, Warrensburg, MO 

Christopher Newport College, Newport News, VA 

David Lipscomb College, Nashville, TN 

Delaware State College, Dover, DE 

East Central University, Ada, OK 

Eastern Washington University, Cheney, WA 

East Tennessee State Univesity, Johnson City, TN 

Freed-Hardeman University, Henderson, TN 

Jacksonville State University, Jacksonville, AL 

Livingstone College, Salisbury, NC 

MacMurray College, Jacksonville, IL 

Missouri Western State College, St. Joseph, MI 

Northeast Louisiana State University, Monroe, LA 
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Northeastern State University, Tahlequah, OK 
Ohio State University-Columbus, Columbus, OH 
Radford University, Radford, VA 
St. Leo (Main Campus), St. Leo, FL 
Southern University, A&M College, Baton Rouge, LA 
Tennessee State University, Nashville, TN 
University of Missouri-St. Louis, St. Louis, MO 
University of North Carolina-Charlotte, Charlotte, NC 
University of Tennessee-Martin, Martin, TN 
University of Wisconsin-River Falls, River Falls, WI 
Upsala College, East Orange, NJ 
Virginia State University, Petersburg, VA 
Wayne State University, Detroit, MI 
William Woods College, Fulton, MO 
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Surveys Sent and Received by Discipline 



Discipline 


Surveys Sent 


Received 


Art 


1,014 


340 


Biology 


1,230 


709 


Chemistry 


1,159 


495 


Communication 


771 


368 


Criminal Justice 


368 


162 


Geology 


409 


303 


History 


1,192 


399 


Literature in English 


335 


136 


Mathematics 


1,216 


537 


Physics 


820 


439 


Political Science 


956 


350 


Psychology 


1,148 


655 


Public Administration 


227 


62 


Social Work 


581 


228 


Sociology 


1,074 


516 


TOTAL 


12,500 


5,699 (45 
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PACAT NATIONAL CURRICULUM SURVEY 

ART 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

♦ ♦ ♦ 



Art Appreciation 
Communications Design 
History of Arts 
Elementary School Art 
Secondary School Art 
Design: Advertising 
Design: Applied 
Design: Environmental 
Design: Fashion 
Design: Interior 
Design: Theatre 
Studio: Ceramics 
Studio: Crafts 
Studio: Drawing 
Studio: Fibers/Textile 
Studio: Glassworking 
Studio: Illustration 
Studio: Jewelry 
Studio: Metal Smithing 
Studio: Painting 
Studio: Papermaking 
Studio: Photography - Film 
Studio: Photography - Video 
Studio: Printmaking 
Studio: Sculpture 
Studio: Water-Color 



76 


18 


46 


56 


9 


36 


276 


6 


20 


77 


11 


45 


74 


10 


40 


51 


9 


48 


88 


16 


16 


3 


2 


13 


3 


0 


6 


13 


1 


13 


3 


0 


12 


106 


9 


123 


30 


6 


36 


272 


12 


32 


30 


5 


60 


1 


0 


9 


42 


4 


47 


24 


4 


64 


15 


5 


44 


196 


6 


92 


10 


2 


26 


86 


5 


101 


12 


2 


46 


140 


8 


114 


150 


12 


103 


55 


3 


74 



5 


75 


17 


103 


9 


56 


16 


158 


2 


5 


1 


30 


0 


79 


7 


121 


1 


68 


9 


138 


6 


56 


24 


146 


0 


13 


13 


194 


3 


13 


25 


281 


2 


13 


13 


303 


0 


37 


10 


266 


0 


33 


7 


285 


6 


54 


4 


38 


3 


60 


6 


199 


0 


11 


1 


12 


3 


53 


17 


172 


0 


34 


4 


292 


10 


59 


18 


160 


11 


47 


20 


170 


13 


26 


18 


219 


0 


30 


3 


13 


11 


61 


43 


187 


3 


55 


6 


84 


2 


39 


12 


227 


3 


45 


5 


25 


5 


40 


3 


27 


14 


79 


37 


78 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 
CONTENT AREA REQUIRED OPTION ELECTIVE 



Calligraphy 


0 


0 


9 


0 


0 

n 


0 

n 


Color Theory/Design 


8 


1 


2 


U 


U 


u 


Studio: Foundations/Fundamentals 


5 


0 


0 


0 


0 


U 


Design - Elements 


9 


0 


1 


0 


0 


0 


Design - 2D 


6 


0 


0 


0 


0 


u 


Design - 3D 


6 


0 


0 


0 


0 


u 

n 


Computer Graphics 


2 


0 


6 


1 


2 


U 


Other 


6 


0 


4 


0 


0 


0 



*** Taught as a separate course 



(Tables based on 340 returns out of 1,014 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 

BIOLOGY 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

♦ ♦ ♦ 



Aquatic Biology 


14 


28 


91 


34 


185 


143 


214 


Biochemistry 


143 


29 


148 


15 


266 


40 


68 


Botany: Field 


49 


57 


127 


50 


155 


102 


169 


Botany: Non-Vascular 


72 


135 


80 


67 


74 


119 


162 


Botany: Vascular 


1 19 


126 


115 


59 


99 


80 


111 


Cellular Biology 


295 


39 


164 


15 


108 


42 


46 


Ecology (inch Field Ecology) 


317 


18 


194 


3 


129 


9 


39 


Embryology 


73 


18 


199 


30 


256 


53 


80 


Entomology 


6 


20 


86 


25 


176 


89 


307 


Evolution (inch Molecular) 


94 


87 


123 


31 


170 


117 


87 


Genetics 


523 


15 


100 


1 


49 


3 


18 


Histology 


14 


9 


117 


21 


275 


73 


200 


Human Anatomy & Physiology 


82 


15 


127 


11 


292 


40 


142 


Microbiology: Bacteriology 


183 


23 


159 


46 


175 


70 


53 


Microbiology: Immunology 


51 


35 


130 


34 


250 


108 


101 


Microbiology: Microbial 
Physiology and Ecology 


28 


47 


55 


65 


81 


209 


224 


Microbiology: Mycology 


16 


39 


49 


61 


104 


168 


272 


Microbiology: Virology 


22 


42 


46 


61 


127 


188 


223 


Molecular Biology 


92 


92 


94 


41 


134 


146 


110 


Physiology: Animal 


136 


40 


216 


18 


160 


71 


68 


Physiology: Cell 


79 


85 


107 


61 


81 


150 


146 


Physiology: Plant 


50 


49 


170 


31 


128 


90 


191 


Zoology: Comparative 
Vertebrate Ariatomy 


81 


14 


168 


25 


225 


61 


135 


Zoology: Field Zoology 


22 


28 


68 


49 


118 


162 


262 


Zoology: Invertebrate Zoology 


110 


69 


162 


15 


161 


62 


130 


Zoology: Vertebrate Zoology 


96 


71 


113 


35 


126 


111 


157 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

♦ * * 



Animal Behavior/Behavioral Biology/ 
Ethology 


1 


0 


10 


0 


27 


0 


Biometrics/Biostatistics/Research Methods 


21 


2 


4 


1 


10 


0 


Developmental Biology 


3 


0 


5 


0 


4 


0 


Parasitology 


1 


0 


8 


0 


24 


0 


Electron Microscopy 


0 


0 


6 


0 


10 


0 


Endocrinology 


0 


0 


7 


0 


14 


2 


Environmental Biology 


2 


0 


4 


0 


9 


0 


Marine Biology 


3 


0 


3 


2 


15 


2 


Neurobiology/Neuroscience/ 

Neuroanatomy/Neurophysiology 


1 


0 


6 


0 


18 


1 

1 


Taxonomy (Animal /Plant) 


3 


0 


4 


0 


2 


Other 


24 


0 


27 


1 


69 


4 



* Taught as a separate course 



(Tables based on 709 returns out of 1,230 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 
CRIMINAL JUSTICE 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

♦ >i< * 



Community Service Systems 


6 


6 


4 


5 


12 


22 


77 


Comparative Criminal Justice 


4 


8 


16 


4 


28 


15 


5 / 

1 Q 


Correction ProcCvSses 


69 


5 


22 


2 


15 


1 


lo 


Crime Prevention 


5 


12 


7 


12 


16 


24 


56 


Criminal Investigation 


25 


1 


25 


4 


27 


10 


40 

on 


Criminal Justice Report Writing 


8 


8 


4 


5 


2 


16 




Criminal Justice System 


64 


14 


3 


0 


4 


18 


29 


Criminal Law 


85 


3 


12 


2 


10 


0 


20 


Criminology 


79 


10 


11 


1 


9 


1 


21 

'IC 


Crisis Intervention 


4 


4 


3 


4 


16 


26 


75 


Deviant Behavior 


28 


10 


19 


3 


23 


19 


30 


Ethics 


16 


13 


15 


2 


8 


28 


50 


History of Justice 


10 


39 


7 


3 


7 


24 


42 


Information Systei^ 
















in Criminal Justice 


5 


11 


9 


7 


3 


15 


82 


Juvenile Justice 


45 


7 


31 


4 


30 


0 


15 


Law Enforcement 


44 


17 


15 


7 


14 


8 


27 


Legal Aspects 


29 


15 


12 


2 


12 


19 


43 


Police Administration 


29 


12 


18 


13 


24 


9 


27 


Police Community Relations 


13 


12 


14 


9 


21 


23 


40 


Probation and Parole 


14 


23 


26 


5 


30 


10 


24 


Research Methods 
















-Criminalistics 


66 


8 


7 


2 


9 


7 


33 


Security Management 


6 


2 


15 


2 


30 


5 


72 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

♦ ♦ * 



Constitutional Law 


3 


0 


2 


0 


2 


0 


Criminal Justice Management 


2 


1 


3 


0 


0 


0 


Gender and Minority Issues 


2 


0 


2 


0 


4 


1 


Research Methods - Other 


5 


0 


2 


0 


1 


0 


Substance Abuse 


1 


0 


2 


0 


3 


0 


Victimology 


1 


0 


1 


0 


4 

A 


0 

n 


Violence 


0 


1 


1 


0 


4 


U 


White Collar and Organized Crime 


0 


0 


3 


0 


7 


1 


Other 


7 


1 


8 


0 


8 


0 



* Taught as a separate course 



(Tables based on 162 returns out of 368 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 
ENGLISH LITERATURE 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 



Adolescent Literature 


8 


0 


11 


4 


26 


11 


75 


American Colonial 


9 


7 


46 


26 


12 


17 


18 


American Modem (1860*s to 
present) 


17 


4 


54 


14 


20 


3 


23 


American Renaissance 


15 


3 


57 


24 


18 


8 


10 


Augustan Period 


15 


8 


67 


13 


16 


8 


8 


Black Writers in Modem 
America 


1 


3 


24 


10 


47 


23 


27 


Biblical Literature 


3 


1 


19 


5 


40 


12 


55 


British Medieval 


20 


10 


73 


9 


14 


4 


5 


British Modem 


16 


6 


68 


10 


23 


8 


4 


British Renaissance 


25 


8 


71 


9 


14 


4 


4 


British Romantic Period 


14 


7 


77 


8 


22 


5 


2 


Chaucer 


9 


3 


77 


11 


23 


8 


4 


Genre: Cinema 


0 


0 


13 


2 


55 


6 


59 


Genre: Drama 


9 


1 


37 


4 


51 


15 


18 


Genre: Fiction 


6 


1 


35 


12 


43 


16 


22 


Genre: Novel 


7 


3 


42 


9 


42 


15 


17 


Genre: Poetry 


18 


2 


37 


5 


42 


14 


17 


Genre: Satire 


0 


1 


13 


8 


21 


41 


51 


Genre: Short Story 


3 


1 


26 


5 


40 


29 


31 


Linguistics: Grammar 


20 


7 


25 


3 


19 


20 


41 


Linguistics: History of the 
Language 

Linguistics: Structure 
Linguistics: Usage 


16 


6 


37 


7 


23 


15 


31 


6 


10 


17 


14 


7 


26 


55 


6 


8 


6 


13 


2 


32 


68 


Literature for Children 


5 


2 


10 


2 


46 


8 


62 


Milton 


4 


5 


68 


15 


18 


17 


8 


Shakespeare 


57 


1 


56 


0 


20 


0 


1 


Southern Literature 


0 


2 


18 


8 


30 


21 


56 


Victorian Literature 


13 


7 


72 


10 


22 


6 


5 


Women in Literature 


1 


2 


27 


10 


65 


11 


19 


World: Greek 


3 


7 


9 


20 


10 


31 


55 


World: Roman 


1 


7 


7 


20 


12 


32 


56 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 
CONTENT AREA REQUIRED OPTION ELECTIVE 



Literary Criticism 


2 


0 


1 


0 


3 


0 


Other 


5 


0 


4 


0 


8 


0 



* Taught as a separate course 



(Tables based on 136 returns out of 335 surveys sent.) 




PACAT - Grant #P116B81711 - Final Report - 32 



35 



PACAT NATIONAL CURRICULUM SURVEY 

GEOLOGY 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA 



REQUIRED 

♦ 



OPTION 



ELECTIVE 

♦ 



BLANK 



Economic Geology 
Engineering Geology 
Environment/Problems 
Field Methods 
Geochemistry 
Geologic Mapping 
Geo morphology 
Geophysics 
Geotectonics 
Historical Geology 
Hydrology 

Igneous & Metamorphic 
Petrology 

Invertebrate Paleontology 
Mineral & Fuel 
Deposit Characteristics 
Mineralogy 
Oceanography 
Optical Mineralogy 
Petrology/Petrography 
Physical Geology 
Planetary Science 
Probability and Statistics 
Sedimentation 
Stratigraphy 
Structural Geology 
Vertebrate Paleontology 
Volcanology 



12 

7 

15 

147 

29 

88 

83 

50 

18 

206 

22 

127 

140 

4 

241 

11 

99 

166 

223 

4 

32 

120 

133 

225 

1 

2 



7 

4 

18 

33 
17 
66 

19 
10 

52 

20 
4 

34 

10 

14 
6 
6 

53 
24 

15 
7 

16 

57 

58 
4 
19 

35 



37 

16 

27 
16 
57 

8 

52 

59 

25 
10 
55 

24 

44 

13 
6 

26 
22 
12 

5 

9 

28 
28 
18 

14 

11 

4 



2 

5 

7 

4 

4 
14 

1 

2 

7 
0 
2 

1 

0 

11 

0 

2 

5 
2 
1 
1 

4 

8 
9 
9 
3 

5 



128 

61 

131 

34 

99 

18 

74 

72 

56 

15 

119 

40 

47 

53 

10 

133 

56 

20 

6 

54 
43 
31 
30 

0 

29 

38 



29 

29 
31 

8 

14 
27 

6 

18 

47 

5 

11 

17 

9 

70 

0 

11 

15 
8 
1 

30 
24 

10 
11 

2 

43 

64 



55 

148 

41 

28 

50 

49 

35 

59 

65 

14 

57 

27 

20 

105 

7 

81 

20 

38 

19 

165 

123 

16 

11 

16 

164 

122 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

♦ ★ * 



Computer Applications 


5 


0 


2 


0 

n 


3 

A 


0 

0 


Glacial Geology 


0 


0 


3 

1 

1 

1 


u 


O 


A 


Marine Geology 
Paleontology 


1 

0 


0 

1 


U 

0 

C\ 


Z 

4 

A 


V 

0 

A 


Petroleum Geology 


1 


0 


U 

r\ 


n 


V 

A 


Photo Geology 


1 


0 


2 

1 

1 

1 


0 


/ 

11 

'X 


O 

A 


Regional Geology 


3 

1 


0 


0 

A 


U 

0 


Remote Sensing 


0 


o 

A 




0 


Seismology 


1 


0 


U 

A 


Z 

A 


0 


X-Ray Crystallography 


2 


0 


L 


U 






Other 


6 


0 


6 


0 


15 


0 



* Taught as a separate course 



(Tables based on 303 returns out of 409 surveys sent. 
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PACAT NATIONAL CURRICULUM SURVEY 

HISTORY 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA 



REQUIRED 

♦ 



OPTION 

♦ 



ELECTIVE 

♦ 



BLANK 



U.S.: Colonial to 1763 
U.S.: 1763 to 1820 
U.S.: 1820 to 1890 
U.S.: 1890 to 1940 
U.S.: 1940 to Present 
U.S.: American Indian/West 
U.S.: Black History 
U.S.: Constitutional 
U.S,: Economic/Urban 
U.S.r Military /Diplomatic 
U.S.: Social/Intellectual/Cultural 
U.S.: Southern 
U.S.: Women 
English: Prior to 1715 
English: 1715 to Present 
English: British Empire 
English: British Constitutional 
European: Ancient 
European: Medieval 
European: Renaissance and 
Reformation 
European: Early Modem 
European: Modem To Present 
European: Church 
European: Economic/Urban 
European: French Revolution 
and Napoleon 
European: Imperialism 
European: Legal 
European: Military/Diplomatic 
European: Social/Intellectual 
/Cultural 

European: Women 

African 

Chinese 

Japanese 

Middle Eastern 

Russian 

Historiography 
Research Methods 



25 


104 


24 


106 


36 


26 


99 


34 


103 


45 


23 


107 


27 


115 


42 


27 


104 


31 


98 


46 


18 


120 


27 


117 


28 


47 


97 


33 


57 


39 


47 


104 


30 


66 


36 


55 


86 


29 


57 


38 


53 


85 


30 


59 


40 


25 


129 


24 


97 


18 


36 


96 


33 


85 


31 


48 


79 


26 


56 


30 


32 


121 


28 


75 


28 


29 


128 


22 


103 


10 


28 


130 


25 


103 


12 


64 


50 


34 


38 


24 


72 


31 


33 


22 


22 


12 


108 


10 


111 


31 


16 


120 


8 


121 


24 


21 


127 


11 


124 


23 


28 


100 


20 


104 


34 


15 


106 


10 


133 


18 


67 


59 


31 


36 


21 


77 


33 


35 


21 


38 


42 


103 


35 


88 


34 


91 


23 


48 


22 


51 


61 


6 


20 


6 


16 


62 


51 


38 


32 


42 


63 


65 


34 


53 


46 


58 


58 


29 


31 


29 


19 


72 


12 


85 


12 


27 


97 


24 


118 


18 


36 


82 


32 


83 


18 


16 


91 


12 


82 


19 


7 


150 


6 


137 


12 


21 


13 


2 


16 


10 


13 


11 


3 


15 


9 



46 

45 

48 

39 

48 

21 

23 

25 

16 

34 

29 

19 

14 
19 
19 

8 

5 

44 

48 

41 

58 

77 

13 

7 

31 

8 
2 
5 

16 

5 

15 
12 
11 

8 

25 

68 

74 



58 

47 

37 

54 
41 

105 

93 

109 

116 

72 

89 

141 

101 

88 

82 

181 

214 

83 

62 

52 

55 
40 

172 

188 

66 

156 

288 

169 

122 

189 

184 

103 

137 

171 

62 

269 

274 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA 



REQUIRED 



OPTION 

♦ 



ELECTIVE 

♦ 



Latin American 


0 


23 


0 


35 


0 


2 

1 

14 


U.S.: State/Regional History 
Other 


0 

0 


17 

69 


0 

3 


9 

57 


0 

0 



* Taught as a separate course 



(Tables based on 399 returns out of 1,192 surveys sent.) 



O 
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PACAT NATIONAL CURRICULUM SURVEY 
MATHEMATICS 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA 



REQUIRED 

♦ 



OPTION 



ELECTIVE 



BLANK 



3-Dimensional Vector Calculus 


208 


175 


17 


14 


22 


30 


Abstract Algebra 


299 


2 


118 


1 


57 


2 


Complex Analysis 


27 


7 


136 


3 


180 


21 
1 A 


Differential Calculus 


296 


162 


1 


1 


4 


lU 


Differential Equations 


177 


13 


131 


5 


147 


5 


Geometry 


78 


2 


149 


3 


197 


11 


Integral Calculus 


293 


178 


2 


1 


1 


3 


Linear Algebra 


412 


14 


40 


2 


14 


0 


Mathematical Models 


34 


26 


90 


17 


109 


63 


Numerical Analysis 


49 


4 


173 


1 


193 


13 


Partial Differential Equations 


6 


20 


48 


28 


101 


80 


Probability 


113 


71 


112 


44 


97 


46 


Real Analysis 


210 


7 


106 


3 


75 


16 


Statistics 


129 


55 


132 


30 


99 


26 


Topology 


13 


7 


92 


6 


158 


26 



71 

58 
163 

63 

59 
97 
59 
55 

198 

104 

254 

54 

120 

66 

235 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA 



REQUIRED OPTION ELECTIVE 

+ + * 



Combinatorics 
Computer Programming 
Discrete Mathematics 
History of Mathematics 
Logic 

Number Theory 
Operations Research 
Other 



2 


0 


11 


0 


16 


12 


0 


5 


0 


5 


25 


6 


12 


0 


10 


13 


4 


11 


0 


12 


3 


6 


4 


0 


5 


8 


4 


17 


0 


16 


3 


0 


10 


0 


11 


24 


7 


51 


1 


29 



* Taught as a separate course 



(Tables based on 537 returns out of 1,216 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 
MASS COMMUNICATION 

CONTENT AREAS E'JCLUDED E'J SURVEY 



CONTENT AREA REQUIRED OPTION 

+ * 



ELECTIVE BLANK 



Advertising 

Business & Professional 
Speaking 

Research Methods 
Debate/Discussion 
Editorial and Feature Writing 
Group Leadership Techniques 
History of Cinema 
History of Radio 
History of Television 
History of Theater 
Interpersonal Communications 
Laws and Ethics 
Mass Communication 
Mass Media and Society 
Mass Media Writing 
News Writing 
Newspaper Editing 
Persuasion 
Production: Cinema 
Production: Electronic Media 
Production : Photography 
Production: Radio 
Production: Television 
Production : Theater 
Public Relations 
Public Speaking 
Reporting 

Theories in Communication 



43 


15 


39 


32 


13 


35 


96 


24 


29 


44 


8 


46 


35 


12 


51 


37 


28 


37 


16 


39 


17 


26 


74 


19 


35 


72 


21 


26 


6 


20 


104 


18 


45 


127 


24 


28 


137 


11 


36 


115 


33 


42 


85 


16 


47 


111 


14 


51 


59 


11 


38 


79 


16 


49 


13 


5 


14 


47 


16 


31 


48 


2 


39 


57 


7 


52 


78 


5 


58 


24 


3 


22 


56 


11 


55 


154 


9 


44 


72 


11 


39 


124 


40 


34 



14 


86 


13 


122 


6 


72 


22 


152 


8 


44 


24 


107 


5 


75 


19 


135 


11 


85 


23 


115 


11 


53 


28 


138 


15 


75 


27 


143 


29 


26 


55 


103 


29 


25 


52 


98 


10 


55 


8 


207 


5 


47 


10 


103 


5 


45 


25 


78 


6 


27 


14 


101 


7 


50 


12 


73 


12 


57 


13 


102 


9 


63 


7 


77 


8 


71 


18 


127 


9 


55 


13 


111 


3 


37 


12 


248 


8 


57 


15 


158 


0 


86 


8 


149 


4 


87 


7 


118 


3 


99 


3 


86 


5 


66 


2 


210 


3 


107 


18 


82 


1 


36 


5 


83 


12 


49 


15 


134 


9 


22 


16 


87 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

♦ + * 



Broadcast Announcing 


1 


0 


3 


0 

r\ 


0 

n 


0 

A 


Creative and Technical Writing 


0 


0 


6 


0 


j 


V/ 

A 


History: Journalism 


0 


1 


2 


0 


0 


U 

A 


History: Mass Media 

Intercultural/Intemational Communications 


3 

3 


0 

0 


0 

4 


0 

0 


0 

10 

1 


U 

0 

A 


Media Criticism 


4 


0 


2 


0 


U 

A 


Media Management/Marketing 


6 


1 


4 

8 


0 


2 

c 


U 


Organizational Communication 


9 


0 


u 

1 


j 

'X 


A 


Print Media Production 


0 


0 


2 


r\ 


V/ 

1 

A 


Visual Design/Graphics 


5 


0 


3 


0 


0 

'1 


Voice & Diclion/Oral Interpretation 


5 


0 


4 


0 




V/ 


Other 


11 


1 


7 


1 


9 


0 



* Taught as a separate course 



(Tables based on 368 returns out of 771 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 

PHYSICS 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

i|c * * 



Astronomy 

Atomic Physics and Spectroscopy 
Condensed Matter Physics 
Electricity 
Fluids 

Heat and Thermodynamics 

Magnetism 

Mechanics 

Nuclear Physics 

Optics: Ray 

Optics: Wave Motion 

Particle Physics 

Physical Electronics 

Quantum Mechanics 

Relativity 

Statistical/Mathematical Physics 



II 


4 


42 


120 


185 


19 


29 


50 


78 


125 


185 


25 


12 


60 


15 


180 


73 


81 


125 


224 


15 


361 


15 


45 


60 


117 


68 


66 


133 


42 


98 


125 


50 


II 


82 


24 


123 


91 


66 


243 


46 


84 


34 


227 


18 


84 


102 


44 



0 


219 


14 


149 


33 


18 


35 


29 


17 


95 


45 


125 


32 


II 


28 


33 


13 


27 


50 


262 


22 


34 


22 


27 


39 


4 


27 


5 


I 


13 


1 


3 


33 


51 


54 


56 


70 


34 


52 


42 


60 


38 


43 


25 


50 


39 


69 


164 


29 


47 


27 


56 


9 


38 


7 


12 


41 


34 


58 


27 


49 


39 


47 


74 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA 



REQUIRED 

* 



OPTION 



ELECTIVE 

♦ 



Digital Electronics 

Analog Electronics 

Computer Applications 

Astrophysics 

Plasma Physics 

Research Methods/Research 

Biophysics 

Acoustics 

Electronic Circuits 

Others 



3 

2 

3 

2 

1 

8 

0 

1 

2 



0 

0 

0 

0 

0 

0 

0 

0 

0 



5 

1 

3 

7 

5 

0 

3 

2 

3 



0 

0 

0 

0 

0 

0 

0 

0 

0 



3 

0 

2 

2 

0 

0 

3 

2 

0 



1 

0 

1 

0 

0 

0 

0 

0 

0 



* Taught as a separate course 



(Tables based on 439 returns out of 820 surveys sent.) 
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BEST COPY AVAILABLE 



PACAT NATIONAL CURRICULUM SURVEY 
POLITICAL SCIENCE 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

i(< >i< >i< 



AMERICAN STUDIES 












20 


21 


American Presidency 


15 


28 


125 


15 


126 


Constitutional Law 


31 


4 


133 


8 


154 


2 


1 8 


Criminal Justice 


2 


8 


21 


12 


73 


63 


171 


Federal Government and Public Policy 


73 


19 


78 


14 


64 


43 


59 


Minority Politics 

Political Parties - Processes and Elections 


5 


8 


34 


17 


58 


79 


149 


18 


11 


127 


15 


140 


18 


21 


Special Interest Groups 


4 


21 


40 


35 


55 


99 


96 


Public Administration/Public Bureaucracy 


32 


4 


112 


6 


122 


19 


55 


Public Law/Judicial Processes 


14 


11 


93 


19 


102 


44 


67 


State/City /Local Government and Policies 


46 


1 


102 


9 


126 


18 


48 


The Congress 


9 


29 


111 


21 


111 


26 


43 


INTERNATIONAL STUDIES 










75 


48 


150 


African Politics 


3 


5 


56 


13 


American Foreign Policy 


18 


4 


133 


8 


143 


14 


30 


Central American Politics 


0 


6 


32 


30 


52 


86 


144 


Comparative Governments 


57 


14 


89 


18 


58 


31 


83 


Comparative Politics 


68 


12 


65 


22 


53 


41 


89 


Far Eastern Politics 


2 


3 


58 


22 


75 


55 


135 


Global Issues 


25 


9 


42 


23 


34 


84 


133 


International Law 


10 


9 


73 


16 


104 


42 


96 


Middle Eastern Politics 


0 


3 


70 


8 


93 


49 


127 


South American Politics 


9 


3 


53 


21 


72 


62 


139 


Soviet Politics 


1 


11 


95 


23 


117 


45 


58 


Third World Politics 


4 


10 


77 


23 


75 


65 


96 


Western European Politics 


10 


10 


109 


19 


90 


47 


65 


METHOD 










17 


53 


267 


Criminal Justice Methods 


2 


1 


5 


5 


Policy Analysis 


16 


7 


52 


18 


50 


59 


148 


Political Analysis 


30 


21 


31 


15 


22 


54 


177 


Quantitative Methods 


66 


29 


37 


13 


43 


43 


1 19 


Research Methods 


99 


23 


39 


15 


47 


27 


100 


NORMATIVE THEORY 










111 


29 


70 


American Political Thought 


20 


10 


99 


11 


Marxist Theory 


1 


23 


40 


38 


48 


94 


106 


Theories of American Politics 


8 


20 


16 


29 


24 


85 


168 


Theories of Political Decision Making 


2 


10 


13 


24 


14 


88 


199 


Theories of Political Organization 


5 


8 


11 


25 


10 


83 


208 


Theories of World Politics (Contemporary) 


14 


16 


45 


29 


39 


72 


135 


Theories of World Politics (Historical) 


6 


11 


16 


34 


15 


79 


189 


Western Political Theory (Contemporary) 


36 


8 


96 


19 


68 


34 


89 


Western Political Theory (Historical) 


64 


5 


128 


11 


75 


20 


47 



"WRITE>1N" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

He ♦ ♦ 



International Organizations 


0 


0 


5 


0 


3 


0 


International Political Economy 


0 


0 


4 


0 


3 


0 


International Relations 


4 


0 


7 


0 


1 


0 


Mass Media/Public Opinion 


0 


0 


6 


0 


7 


0 


U.S, Military, Defense, and National Security 


0 


0 


5 


0 


7 


0 


Women in Politics 


0 


0 


4 


1 


7 


0 


Other 


2 


0 


16 


0 


35 


0 



* Taught as a separate course (Tables based on 350 returns out of 956 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 

PSYCHOLOGY 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

+ ♦ ♦ 



Abnormal 


110 


1 


301 


1 


189 


1 


11 


Animal Leaming/Motivation 


55 


61 


204 


42 


66 


69 


117 

1 1 a 


Clinical/Counseling 

Community 

Comparative 

Developmental 

Educational 


35 

4 

5 

139 

18 


10 

11 

20 

3 

4 


147 

33 

84 

278 

50 


13 

11 

25 

1 

2 


235 

57 

64 

163 

172 


61 

88 

80 

6 

29 


1 13 
410 
336 
24 
339 


Experimental Design 
Forensic 


452 

3 


57 

3 


32 

12 


3 

8 


30 

48 


11 

46 


29 

494 


Gender 


5 


13 


52 


24 


198 


102 


220 


History and Systems 


216 


18 


133 


6 


151 


20 


70 

.4C 


Human Learning /Cognition 
Industrial 


109 

11 


31 

4 


281 

126 


21 

6 


99 

247 


28 

28 


45 

192 


Minority 

Personality 

Physiological 

Psycholinguistics 

Sensation and Perception 


4 

125 

107 

4 

22 


9 
7 

10 
12 
38 


20 

295 

309 

52 

251 


18 

3 

3 

23 

20 


68 

152 

117 

68 

96 


82 

3 

14 

82 

54 


413 

29 

54 

373 

133 


Social 


100 


2 


304 


5 


172 


0 


31 


Statistics 


502 


36 


28 


1 


12 


4 


31 


Testing and Measurement 


86 


6 


196 


10 


205 


31 


80 



''WRITE-IN'' CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA REQUIRED OPTION ELECTIVE 

♦ ♦ ♦ 



Adjustment 


2 


0 


5 


0 


22 

8 

21 


0 

n 


Applied 

Behavior Modification 


1 

0 


0 

0 


2 

12 


U 

0 


u 

0 


Computer Analysis/Use/ Application 


4 


0 


5 


0 


7 

8 


0 


Cross/Multi Cultural 


0 


0 


5 


0 


u 

1 


Emotion, Stress and Coping 


0 


0 


3 


0 


10 


Environmental 


1 


0 


5 


0 

1 


14 


U 

r\ 


Ethics/Professional Conduct 


4 


0 


1 


3 


U 

1 

n 


Exceptionalities 


2 


0 


11 

12 


0 

n 


21 

18 

16 

27 


Group Dynamics 


4 


0 


U 


1 

n 


Health/Sports 


0 


0 


12 


U 


Human Sexuality 


2 


0 


9 


U 


u 

n 


Humanistic 


0 


0 


4 


0 


9 


U 


Psychopharmacology 

Religion 


2 

1 


0 

0 


12 

4 


0 

0 


24 

13 


0 

0 


Other 


4 


1 


29 


0 


53 


0 



* Taught as a separate course 



(Tables based on 655 returns out of 1, 148 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 
PUBLIC ADMINISTRATION 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA REQUIRED OPTION ELECTIVE BLANK 

♦ + * 



Economic Aspects 
Ethics 

Governmental Relations 
Human Resources 
International & 

Regional Development 
International Organizations 
Judicial Processes 
Legal Aspects 
Legislative Processes 
Local Government 
National Government 
Organizational Dynamics 
Planning 

Public Administration 
Public Policy Analysis 
Research Methods 
State Government 
Statistics 
Urban Affairs 



32 

8 

16 

21 



2 

18 

15 

8 



7 

2 

6 

12 



7 

7 

2 

3 



4 
8 

5 
7 



12 

7 

4 



7 

11 

7 



1 

1 

8 

10 

7 

19 
34 

20 

5 

42 

26 

34 

16 

36 

6 



1 

0 

5 

8 

17 

11 

11 

11 

8 

5 

2 

4 

14 

7 

2 



4 

3 
11 

13 
11 

14 

4 

7 
11 

8 
13 

5 

12 

3 

15 



4 

4 

3 

5 
1 

4 
0 

5 

5 
0 
4 
4 

6 
1 
4 



8 

14 
12 

8 

10 

8 

5 

5 

7 

3 

4 
3 
9 
3 

15 



15 

7 
9 
9 
6 
2 
3 

8 
11 

0 

3 

0 

1 

2 

6 



29 

33 

14 

9 
10 

4 

5 

6 

15 
4 

10 
12 

4 

10 

14 



"WRITE-IN" CONTENT AREAS 



CONTENT AREA REQUIRED OPTION 



Budgets/ Accounting & Finance 


8 


0 


0 


0 

1 

n 


0 

n 


0 

0 


Computer Applications 


3 

1 


0 


u 


vl 

0 


0 


Management 


0 


* 




vl 




Other 


3 


0 


0 


0 


0 


0 



(* Taught as a separate course) 
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PACAT NATIONAL CURRICULUM SURVEY 

SOCIAL WORK 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA 



Afro-American Family 
Child and Adolescent Behavior 
Current Issues in Social Work 
Family Planning and 
Population Policy 
Health Care Delivery 
Human Sexuality 
Legal Aspects of Social Work 
Methods of Intervention 
Policy-Program Analysis 
Quantitative Methods/ 
Statistics 
Research Methods 
Social Services for the Abused 
and Neglected Child 
Social Services for the 
Child and Family 
Social Services for 
Health and Rehabilitation 
Social Services for Mental 
Health and Retardation 
(§)Human Behavior in the 
Social Environment 
Social Services for the Aged 
Social Welfare Policy 
and Service 

Social Work Practice Methods 
Social Work Skills 



REQUIRED 

♦ 


OPTION 

♦ 


ELEC 

♦ 


TIVE 


BLANK 


21 


92 


4 


10 


4 


38 


31 


44 


91 


11 


2 


11 


15 


26 


32 


92 


0 


6 


10 


29 


31 


8 


54 


6 


10 


8 


35 


79 


9 


68 


14 


7 


26 


30 


46 


21 


76 


4 


4 


41 


25 


29 


18 


62 


7 


8 


16 


33 


56 


132 


33 


2 


0 


4 


6 


23 


90 


61 


1 


4 


5 


11 


28 


114 


24 


10 


1 


22 


5 


24 


159 


13 


1 


2 


9 


2 


14 


21 


62 


17 


10 


34 


23 


33 


30 


64 


18 


9 


29 


23 


27 


7 


72 


7 


14 


22 


32 


46 


7 


77 


15 


9 


13 


37 


42 


86 


74 


1 


3 


1 


10 


25 


21 


59 


22 


7 


37 


26 


28 


141 


27 


3 


1 


6 


4 


18 


150 


22 


3 


1 


5 


2 


17 


125 


42 


‘ 


1 


3 


4 


24 



(§) This area was inadvertently listed as "Social Environment" in the surveys. The incorrect listing may account 
for some departments indicating that they do not require it when in fact they do. 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA 



REQUIRED 



OPTION 



ELECTIVE 

* 



Minority Issues 
Ethics 

Community Organization/Analysis 


3 

2 

3 


1 

3 

0 


2 

0 

0 


1 

0 

0 


1 

1 

0 


0 

0 

0 


Other 


4 


1 


6 


0 


10 


0 



♦ Taught as a separate course 



(Tables based on 228 returns out of 581 surveys sent.) 
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PACAT NATIONAL CURRICULUM SURVEY 

SOCIOLOGY 

CONTENT AREAS INCLUDED IN SURVEY 



CONTENT AREA 



REQUIRED 



OPTION 



ELECTIVE 



BLANK 



Behavior in Organization 

Criminology 

Deviance/Corrections 

Juvenile Delinquency 

Marriage and Family 

Medical Sociology 

Race and Ethnic Relations 

Research Methods 

Social Anthropology 

Social Change 

Social Problems 

Social Stratification 

Social Theory/Thought 

Society and the Individual 

Sociology of Aged and Aging 

Sociology of Sex Roles 

Sociology of Work 

Statistics 

Urban Sociology 



23 
34 

24 
18 
32 

4 

38 

467 

80 

18 

94 

42 

447 

56 

9 

4 

4 

350 

22 



7 

1 

0 

3 
0 
0 

4 
6 
4 
7 

4 
11 

6 

12 

1 

5 

2 

19 

2 



63 
108 

93 

66 

no 

51 

109 

11 

46 

49 
89 

105 

8 

64 
69 

50 
45 
25 
89 



13 

5 

10 

16 

2 

1 

7 

1 

0 

14 

1 

9 
2 
9 
2 

10 

6 
2 
3 



176 

293 

268 

220 

338 

245 

297 

21 

165 

170 

244 

212 

30 

141 

257 

250 

173 

46 

259 



135 

31 

72 

no 

10 

35 

34 

4 
48 

147 

43 

91 

5 

111 

62 

108 

118 

28 

65 



99 

44 

49 

83 

24 

180 

27 

6 

173 

111 

41 

46 

18 

123 

116 

89 

168 

46 

76 



"WRITE-IN" CONTENT AREAS 

(Write-in content area names are based upon nomenclature provided by respondents.) 



CONTENT AREA 



REQUIRED OPTION ELECTIVE 

+ + + 



Applied Sociology 


0 


0 


4 

c 


1 


13 

17 

10 

13 

71 


0 

0 


Cross Cultural Sociology /Non U.S. Systems 


3 


0 


D 


u 

c\ 


\j 

0 


Culture/Social Institutions (U.S.) 
Demography 


3 

4 


0 

0 

r\ 


6 

4 


U 

0 

n 


\J 

0 

0 


Political Sociology 


0 

1 


0 

r\ 


0 

8 

o 


n 


Zf 1 

71 


0 


Population Issues 


0 

r\ 


n 


Q 


0 


Social Psychology 

Sociology of Communication/Mass Media 
Sociology of Education 
Sociology of Religion 


12 

0 

0 

0 


0 

0 

0 

0 

r\ 


y 

6 

5 

18 


\3 

0 

0 

0 

n 


10 

16 

36 

21 


0 

0 

0 

0 


Sociology of the Law 


0 


0 


O 


V/ 




Other 


12 


0 


53 


0 


83 


2 



* Taught as a separate course 



(Tables based on 516 returns out of 1,074 surveys sent.) 
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BEST COPY AVAILABLE 



PACAT ART CURRICULUM PATTERN A 

(Of the 297 classifiable surveys returned, 174 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 7) 



Drawing 
History of Arts 
Painting 



Advertising 
Art Appreciation 
Ceramics 
Crafts 

Elementary School Art 
Photography - Film 
Printmakmg 
Sculpture 

Secondary School Art 
Water-Color 



PACAT ART CURRICULUM PATTERN B 

(Of the 297 classifiable surveys returned, 63 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 4) 

Drawing Ceramics 

History of Arts Communications Design 

Painting Photography - Film 

Printmaking 

Sculpture 

Water-Color 



PACAT ART CURRICULUM PATTEIW C 

(Of the 297 classifiable surveys relumed, 12 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 

Drawing Aft Appreciation 

History of Arts Ceramics 

Sculpture Water-Color 



PACAT ART CURRICULUM PATTERN D 

(Of the 297 classifiable surveys returned, 17 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 

Drawing Applied 

Painting Art Appreciation 

Printmaking Communications Design 

Sculpture 



PACAT ART CURRICULUM PATTERN E 

(Of the 297 classifiable surveys returned, 31 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 

Drawing Painting 

History of Arts Printmaking 

Sculpture 
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PACAT BIOLOGY CURRICULUM PATTERN A 

(Of the 591 classifiable surveys returned, 217 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 6) 


Cellular Biology 

Ecology (incluaing Field Ecology) 

Genetics 

Vascular Botany 


Animal Physiology 
Bacteriology 
Biochemistir 
Cell Physiology 
Embryology 

Evolution (including Molecular) 
Invertebrate & Vertebrate Zoology 
Molecular Biology 
Non-Vascular Botany 


PACAT BIOLOGY CURRICULUM PATTERN B 

(Of the 591 classifiable surveys returned, 150 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 2) 


Cellular Biology 

Ecology (incluaing Field Ecology) 
Genetics 


Animal & Plant Physiology 

Bacteriology 

Embryology 


PACAT BIOLOGY CURRICULUM PATTERN C 
(Of the 591 classifiable surveys returned, 91 fell within this curriculum.) 

(A * indicates areas which were added to the original pattern at the request of participating departments.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Ecology (including Field Ecology) 
Genetics 


Animal Physiology (*) 

Bacteriology 
Biochemistry 
Cellular Biology (*) 

Embryology 

Evolution (including Molecular) 

Human Anatomy & Physiology 
Immunology 
Invertebrate Zoology 
Molecular Biology 
Plant Physiology (*) 

Vascular and Non-vascular Botany (*) 
Vertebrate and Invertebrate Zoology (*) 


PACAT BIOLOGY CURRICULUM PATTERN D 

(Of the 591 classifiable surveys returned, 42 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 4) 


Cellular Biology 
Genetics 


Bacteriology 
Biochemistry 
Cell Physiology 
Immunology 
Molecular Biology 
Virology 
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PACAT BIOLOGY CURRICULUM PATTERN E 

(Of the 591 classifiable surveys returned, 24 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 3) 



Cellular Biology 
Genetics 


Animal Physiology 
Bacteriology 

Biometrics/Statistics/Research Methods 
and Design 

Ecology (including Field Ecology) 
Human Anatomy & Physiology 



PACAT BIOLOGY CURRICULUM PATTERN F 

(Of the 591 classifiable surveys returned, 25 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 3) 



Genetics 
Vascular Botany 


Biochemistry 

Comparative Vertebrate Anatomy 
Field Botany 

Invertebrate & Vertebrate Zoology 
Non-Vascular Botany 



PACAT BIOLOGY CURRICULUM PATTERN G 

(Of the 591 classifiable surveys returned, 24 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 



Bacteriology 

Genetics 


Animal Physiology 
Biochemistry 
Embryology 
Immunology 



PACAT BIOLOGY CURRICULUM PATTERN H 

(Of the 591 classifiable surveys returned, 18 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 



Ecology (including Field Ecology) 
Genetics 


Animal Physiology 
Non-Vascular Botany 
Vascular Botany 



ERIC 
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PACAT CRIMINAL JUSTICE CURRICULUM PATTERN A 

(Of the 96 classifiable surveys returned, 52 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 7) 



Correction Processes 
Criminal Law 
Criminology 



Criminal Justice Systems 

Deviant Behavior 

Histo^ of Justice 

Juvenile Justice 

Law Enforcement 

Legal Aspects 

Police Administration 

Police Community Relations 

Probation and Parole 

Research Methods - Criminalistics 



PACAT CRIMINAL JUSTICE CURRICULUM PATTERN B 

(Of the 96 classifiable surveys returned, 19 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 3) 



Correction Processes 
Criminal Justice Systems 
Law Enforcement 



Criminology 

Deviant Behavior 

Juvenile Justice 

Police Administration 

Research Methods - Criminalistics 



PACAT CRIMINAL JUSTICE CURRICULUM PATTERN C 

(Of the 96 classifiable surveys returned, 25 fell within this curriculum.) 



COMMON CONTENT AREAS 

Criminal Law 
Criminology 



Optional Content Areas (Select 2) 

Correction Processes 
Criminal Investigation 
Juvenile Justice 



NOTE: These are preliminary patterns. Although comments are always appreciated, we 
particularly would like to receive evaluations and suggested revisions pnor to January 30, 
1993. After this date, we will be actively seeking to build a national consortium ot 
departments for the purpose of outcomes assessment using the patterns above. Please send 
comments and suggestions to: 



PACAT 

Austin Peay State University 
P.O. Box 4568 
Clarksville, TN 37044 
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PACAT LITERATURE IN ENGLISH CURRICULUM PATTERN A 

(Of the 93 classifiable surveys returned, 88 fell within this curriculum.) 



COMMON CONTENT AREAS 

Renaissance 

Shakespeare 



Optional Content Areas (Select 8) 
American Colonial 

American Modem (1860's to present) 

American Renaissance 

Augustan Period 

British Medieval 

British Modem 

British Romantic Period 

Chaucer 

Genre: Drama 

Genre: Poetry 

Victorian Literature 



PACAT LITERATURE IN ENGLISH CURRICULUM PATTERN B 

(Of the 93 classifiable surveys returned, 5 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 4) 



British Renaissance 
Shakespeare 



American Colonial 
American Renaissance 
Biblical Literature 
British Medieval 
Chaucer 



PACAT LITERATURE IN ENGLISH CURRICULUM PATTERN C 

(This pattern serves as the basis of the Area Concentration Achievement Test developed independently 
in 1989 and does not conform to the same structural divisions as those above.) 



CONTENT AREAS 
American to 1865 

American Modern (1860's to present) 
British Medieval 
British Renaissance 
British Romantic Period 
British Victorian Literature 
Shakespeare 

Linguistics (Grammar, History, Structure, Usage) 
Restoration/ 1 8th Century/PreRomantic 
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PACAT GEOLOGY CURRICULUM PATTERN A 

(Of the 245 classifiable surveys returned, 165 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Historical Geology 
Mineralogy 
Physical Geology 
Stratigraphy 
Structural Geology 


Field Methods 
Geologic Mapping 
Geomorphology 

Igneous and Metamorohic Petrology 
Invertebrate Paleontology 
Optical Mineralogy 
Petrology/Petrography 
Sedimentation 


PACAT GEOLOGY CURRICULUM PATTERN B 
(Of the 245 classifiable surveys returned, 41 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 7) 


Historical Geology 
Mineralogy 
Physical Geology 


Environment/Problems 
Field Methods 
Geomorphology 

Igneous and Metamorphic Petrology 

Invertebrate Paleontology 

Optical Mineralogy 

Petrology/Petrography 

Sedimentation 

Stratigraphy 

Structural Geology 


PACAT GEOLOGY CURRICULUM PATTERN C 

(Of the 245 classifiable surveys returned, 21 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 7) 


Mineralogy 
Petrology/Petrography 
Structural Geology 


Economic Geology 
Environment/Problems 
Field Methods 
Geochemistry 
Geomorphology 

Igneous and Metamorohic Petrology 
Invertebrate Paleontology 
Optical Mineralogy 
Sedimentation 
Stratigraphy 



PACAT GEOLOGY CURRICULUM PATTERN D 

(Of the 245 classifiable surveys returned, 18 fell within this curriculum.) 

Optional Content Areas (Select 2) 



COMMON CONTENT AREAS 

Historical Geology 
Physical Geology 



Geomorphology 
Mineralogy 
Structural Geology 



er|c 
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PACAT HISTORY CURRICULUM PATTERN A 

(Of the 219 classifiable surveys returned, 141 fell within this curriculum.) 



COMMON CONTENT AREAS 



European: 

European: 

European: 



Medieval 
Early Modem 
Modern To Present 



Optional Content Areas (Select 7) 

U.S.: Colonial to 1763 
U.S.: 1763 to 1820 
U.S.: 1820 to 1890 
U.S.: 1890 to 1940 
U.S.: 1940 to Present 
U.S.: Black History 
U.S.: Military/Diplomatic 
U.S.: Social/Intellectual/Cultural 
European: Ancient 
European: French Revolution and 
Napoleon 

European: Renaissance and Reformation 
Non-European: African 
Non-European: China, Japan, India 
Historiography 



PACAT HISTORY CURRICULUM PATTERN B 

(Of the 219 classifiable surveys returned, 53 fell within this curriculum.) 

Optional Content Areas (Select 4) 



COMMON CONTENT AREAS 



U.S.: 

U.S.: 



1763 to 1820 
1820 to 1890 



U.S.: Colonial to 1763 

U.S.: 1890 to 1940 

U.S.: 1940 to Present 

U.S.: American Indian/Western U.S. 

U.S.: Black History 

U.S.: Military/Diplomatic 

U.S.: Southern 



PACAT HISTORY CURRICULUM PATTERN C 

(Of the 219 classifiable surveys returned, 25 fell within this curriculum.) 



COMMON CONTENT AREAS 



lods for History 



Optional Content Areas (Select 2) 

U.S.: 1890 to 1940 
European: Ancient 

European: Renaissance and Reformation 



ERIC 
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PACAT MATHEMATICS CURRICULUM PATTERN A 

(Of the 428 classifiable surveys returned, 294 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


3-Dimensional Vector Calculus 


Complex Analysis 


Abstract Algebra 


Differential Equations 


Differential Calculus 


Geometry 


Integral Calculus 


Mathematical Models 


Linear Algebra 


Numerical Analysis 

Partial Differential Equations 

Probability 

Real Analysis 

Statistics 



PACAT MATHEMATICS CURRICULl^ PATTERN B 

(Of the 428 classifiable surveys returned, 54 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


3-Dimensional Vector Calculus 
Differential Calculus 
Integral Calculus 
Linear Algebra 


Abstract Algebra 
Complex Analysis 
Differential Equations 
Geometry 

Mathematical Models 

Numerical Analysis 

Partial Differential Equations 

Probability 

Real Analysis 

Statistics 


PACAT MATHEMATICS CURRICULA PATTERN C 

(Of the 428 classifiable surveys relumed, 80 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 8) 


Differential Calculus 
Integral Calculus 


Abnormal 
Abstract Algebra 
Complex Analysis 
Differential Equations 
Geometry 
Linear Algebra 
Mathematical Models 
Numerical Analysis 
Probability 
Real Analysis 
Statistics 
Topology 



NOTE: These are preliminary patterns. Although comments are always appreciated, we 
particularly would like to receive evaluations and suggested revisions prior to January 30, 
1993. Please send comments and suggestions to: 

PACAT 

Austin Peay State University 
P.O. Box 4568 
Clarksville, TN 37044 
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PACAT MASS COMMUNICATION CURRICULUM PATTERN A 

(Of the 253 classifiable surveys returned, 75 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 8) 


Public Speaking 
Theories in Communication 


Communications Research Methods 

Group Leadership Techniques 

History of Cinema 

History of Radio 

History of Television 

Interpersonal Communications 

Laws and Ethics 

Mass Communication 

Mass Media and Society 

News Writing 

Persuasion 



PACAT MASS COMMUNICATION CURRICULUM PATTERN B 

(Of the 253 classifiable surveys returned, 66 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 8) 


Laws and Ethics 
News Writing 


Advertising 

Editorial and Feature Writing 
History of Television 
Mass Media and Society 
Mass Media Writing 
Newspaper Editing 
Production: Photography 
Production: Radio 
Production: Television 
Public Relations 
Reporting 


PACAT MASS COMMUNICATION CURRICULUM PATTERN C 

(Of the 253 classifiable surveys returned, 16 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 6) 


Communications Research Methods 
Theories in Communication 


History of Cinema 
History of Radio 
History of Television 
Laws and Ethics 
Mass Media and Society 
Mass Media Writing 
Production: Electronic Media 
Production: Radio 
Production: Television 
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BEST COPY AVAILABLE 



PACAT MASS COMMUNICATION CURRICULUM PATTERN D 

(Of the 253 classifiable surveys returned, 47 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Interpersonal Communications 
Public Speaking 


Debate/D iscussion 
Group Leadership Techniques 
History of Theater 
Mass Communication 
Mass Media and Society 
Persuasion 
Production: Theater 
Theories in Communication 



PACAT MASS COMMUNICATION CURRICULUM PATTERN E 

(Of the 253 classifiable surveys returned, 12 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 4) 



Communications Research Methods 
Mass Communication 


History of Theater 
Interpersonal Communications 
Mass Media Writing 
Persuasion 

Production: Television 
Public Speaking 



PACAT MASS COMMUNICATION CURRICULA PATTERN F 

(Of the 253 classifiable surveys returned, 15 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 3) 



Laws and Ethics 
News Writing 


Communications Research Methods 
Editorial and Feature Writing 
Mass Media and Society 
Newsp^r Editing 
Public Relations 



PACAT MASS COMMUNICATION CURRICULUM PATTERN G 

(Of the 253 classifiable surveys returned, 10 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 



Laws and Ethics 
Mass Media and Society 


Advertising 
Mass Media Writing 
News Writing 



PACAT MASS COMMUNICATION CURRICULUM PATTERN H 

(Of the 253 classifiable surveys returned, 12 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 2) 



Communications Research Methods 
Theories in Communication 


Debate/Discussion 
Interpersonal Communications 
Persuasion 
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PACAT PHYSICS CURRICULUM PATTERN A 

(Of the 359 classifiable surveys returned, 174 fell within this curriculum.) 



COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Atomic Physics and Spectroscopy 

Electricity (Alternating Current) 

Magnetism 

Mechanics 

Quantum Mechanics 


Condensed Matter Physics 

Heat and Thermodynamics 

Nuclear Physics 

Optics: Ray 

Optics: Wave Motion 

Physical Electronics (Direct Current) 

Relativity 

Statistical/Mathematical Physics 


PACAT PHYSICS CURRICULUM PATTERN B 
(Of the 359 classifiable surveys returned, 63 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Mechanics 
Magnetism 
Quantum Mechanics 
Electricity (Alternating Current) 
Heat and Thermodynamics 


Astronomy 
Nuclear Physics 
Optics: Ray 
Optics: Wave Motion 
Particle Physics 

Physical Electronics (Direct Current) 
Relativity 

Statistical/Mathematical Physics 


PACAT PHYSICS CURRICULUM PATTERN C 

(Of the 359 classifiable surveys returned, 12 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 7) 


Atomic Physics and Spectroscopy 

Magnetism 

Mechanics 


Astronomy 

Electricity (Alternating Current) 

Heat and Thermodynamics 

Nuclear Physics 

Optics: Ray 

Optics: Wave Motion 

Physical Electronics (Direct Current) 

Quantum Mechanics 

Relativity 

Statistical/Mathematical Physics 


PACAT PHYSICS CURRICULUM PATTERN D 

(Of the 359 classifiable surveys returned, 17 fell within this curriculum.) 


COMMON CONTENT AREAS 


Optional Content Areas (Select 5) 


Electricity (Alternating Current) 

Magnetism 

Mechanics 


Astronomy 

Condensed Matter Physics 

Heat and Thermodynamics 

Optics: Ray 

Optics: Wave Motion 

Physical Electronics (Direct Current) 

Quantum Mechanics 

Relativity 
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PACAT PHYSICS CURRICULUM PATTERN E 

(Of the 359 classifiable surveys returned, 31 fell within this curriculum.) 



COMMON CONTENT AREAS 

Atomic Physics and Spectroscopy 

Mechanics 

Quantum Mechanics 



Optional Content Areas (Select 2) 

Electricity (Alternating Current) 

Magnetism 

Nuclear Physics 

Relativity 
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PACAT PSYCHOLOGY CURRICULUM PATTER A 

(Of the 585 classifiable surveys relumed, 140 fell within this curriculum.) 



COMMON CONTENT AREAS 
Abnormal 

Animal Learning and Motivation 

Experimental Design 

Human Learning and Cognition 

Personality 

Physiological 

Social 

Statistics 


Optional Content Areas (Select 2) 
Clinical and Counseling 
Developmental 
History and Systems 
Sensation and Perception 
Testing and Measurement 


PACAT PSYCHOLOGY CURRICULUM PATTER B 

(Of the 585 classifiable surveys returned, 95 fell within this curriculum.) 


COMMON CONTENT AREAS 
Abnormal 
Developmental 
Experimental Design 
History and Systems 
Personality 
Statistics 


Optional Content Areas (Select 2) 
Animal Learning and Motivation 
Clinical and Counseling 
Human Learning and Cognition 
Physiological 
Sensation and Perception 
Social 

Testing and Measurement 


PACAT PSYCHOLOGY CURRICULUM PATTERN C 

(Of the 585 classifiable surveys returned, 92 fell within this curriculum.) 


COMMON CONTENT AREAS 
Developmental 
Experimental Design 
Social 
Statistics 


Optional Content Areas (Select 2) 
Abnormal 

Clinical and Counseling 

History and Systems 

Human Learning and Cognition 

Personality 

Physiological 

Testing and Measurement 



PACAT PSYCHOLOGY CURRICULUM PATTERN D 

(Of the 585 classifiable surveys returned, 43 fell within this curriculum.) 



COMMON CONTENT AREAS 
Experimental Design 
History and Systems 
Statistics 



Optional Content Areas (Select 1) 
Abnormal 

Clinical and Counseling 
Developmental 

Human Learning and Cognition 

Personality 

Physiological 

Social 

Testing and Measurement 
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PACAT PSYCHOLOGY CURRICULUM PATTERN E 

(Of the 585 classifiable surveys returned, 53 fell within this curriculum.) 



COMMON CONTENT AREAS 
Experimental Design 
Statistics 



Optional Content Areas (Select 2) 
Abnormal 

Animal Learning and Motivation 
Developmental 

Human Learning and Cognition 

Personality 

Physiological 

Sensation and Perception 



PACAT PSYCHOLOGY CURRICULUM PATTERN F 

(Of the 585 classifiable surveys returned, 31 fell within this curriculum.) 



COMMON CONTENT AREA 
Statistics 



Optional Content Areas (Select 3) 
History and Systems 
Human Learning and Cognition 
Personality 
Physiological 
Social 



PACAT PSYCHOLOGY CURRICULUM PATTER G 

(Of the 585 classifiable surveys returned, 32 fell within this curriculum.) 



COMMON CONTENT AREAS 
Abnormal 
Developmental 
Social 



Optional Content Areas (Select 3) 
Animal Learning and Motivation 
Clinical and Counseling 
Experimental Design 
Human Learning and Cognition 
Personality 
Physiological 
Testing and Measurement 



PACAT PSYCHOLOGY CURRICULUM PATTERN H 

(Of the 585 classifiable surveys returned, 9 fell within this curriculum.) 



COMMON CONTENT AREAS 
Abnormal 

Experimental Design 



Optional Content Areas (Select 2) 
Animal Learning and Motivation 
Clinical and Counseling 
Developmental 
History and Systems 
Statistics 

Testing and Measurement 



PACAT PSYCHOLOGY CURRICULUM PATTERN I 

(Of the 585 classifiable surveys returned, 90 fell within this curriculum.) 



COMMON CONTENT AREAS 
Experimental Design 
Human Learning and Cognition 
Statistics 



Optional Content Areas (Select 3) 
Abnormal P^chology 
History and Systems of Psychology 
Personality 

Physiological Psychology 
Testing and Measurement 
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BEST COPY AVAILABLE 



PACAT SOCIOLOGY CURRICULUM PATTERN A 

(Of the 462 classifiable surveys returned, 379 fell within this curriculum.) 

COMMON CONTENT AREAS Optional Content Areas (Select 7) 



Research Methods Criminology 

Social Theory/Thought Deviance and Corrections 

Statistics Marriage and Family 

Race and Ethnic Relations 
Social Anthropology 
Social Change 
Social Problems 
Social Stratification 
Society and the Individual 
(Combined Areas) Medical Sociology, 
Sociology of the Aged, and Urban 
Sociology 



PACAT SOCIOLOGY CURRICULUM PATTERN B 

(Of the 462 classifiable surveys returned, 61 fell within this curriculum.) 

[As in Curriculum Pattern A above but the department selects 5 options.] 



PACAT SOCIOLOGY CURRICULUM PATTERN C 

(Of the 462 classifiable surveys relumed, 22 fell within this curriculum.) 

[As in Curriculum Pattern A above but the department selects 2 options.] 



NOTE: These are preliminary patterns. Although comments are always appreciated, we 
particularly would uke to receive evaluations and suggested revisions prior to January 30, 
1993. After this date, we will be actively seeking to build a national consortium of 
departments for the purpose of outcomes assessment using the patterns above. Please send 
comments and suggestions to: 



PACAT 

Austin Peay State University 
P.O. Box 4568 
Clarksville, TN 37044 
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PACAT SOCIAL WORK CURRICULUM PATTERN A 

(Of the 153 classifiable surveys returned, 119 fell within this curriculum.) 



COMMON CONTENT AREAS 

Human Behavior in the Social 
Environment 
Methods of Intervention 
Policy-Program Analysis 
Research Methods 
Social Welfare Policy and Services 
Social Work Practice Methods 
Social Work Skills 



Optional Content Areas (Select 3) 

Afro-American Family 
Child and Adolescent Behavior 
Current Issues in Social Work 
Quantitative Methods/Statistics 
Social Services for Child and Family 



PACAT SOCIAL WORK CURRICULUM PATTERN B 

(Of the 153 classifiable surveys returned, 34 fell within this curriculum.) 



COMMON CONTENT AREAS 

Social Welfare Policy and Services 
Social Work Practice Methods 
Social Work Skills 



Optional Content Areas (Select 5) 

Child and Adolescent Behavior 
Current Issues in Social Work 
Human Behavior in the Social 
Environment 
Human Sexuality 
Methods of Intervention 
Policy-Program Analysis 
Quantitative Methods/Statistics 
Research Methods 



PACAT SOCIAL WORK CURRICULUM PATTERN C 

(This pattern serves as the basis of the Area Concentration Achievement Test developed by a 
consortium of Tennessee Social Work departments in 1984 and does not conform to the same 

structural divisions as those above.) 

CONTENT AREAS 

Human Behavior in the Social Environment 
Policy 
Practice 
Research 
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APPENDIX V: Publications, Presentations and Workshops 
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Publications by PACAT Staff (1988-92) 



Golden, A. and Squire, D. (1991). The third alternative. Assessment Update, 3 (2), 10-11. 

Golden, A. (1991). PACAT: A national project in cooperative major field assessment. In 
R. McCormick (Ed.), The 1990 Montclair assessment conference: Strategies and 
prospects for the decade (pp. 80-83). Upper Montclair, NJ: Montclair State College. 

Golden, A. (1989). Project for Area Concentration Achievement Testing. Washington, DC: 
AASCU/ERIC Model Programs Inventory Project. (ERIC Document Reproduction 
Service No. ED 306 863). 



Other Publications Concerning PACAT 

Cone, A. (1988). Low tech/high touch criterion-based learning. Psychological Reports, 63, 
203-207. 

Cone, A. (1990). Frequency of testing in criterion-based learning. Psychological Reports, 
67, 396-398. 



Presentations by PACAT Staff (1988-92) 



Golden, A., Carter, G., and Ferraro, E. (1992, March). The AC AT: A Model for 

Consortium Based Outcomes Assessment in Social Work. Presented at the meeting of 
the Council for Social Work Education. Kansas City, MO. 

Rogers, R. (Moderator), Beers, S., Golden, A., Goldstein, M., and Patterson, D. (1992, 
March). CTUP Roundtable: Assessment of the Undergraduate Psychology Program. 
Roundtable presentation at the meeting of the Southeastern Psychological Association. 
Knoxville, TN. 

Lutzer, D. (Moderator), Bell, D., Fletcher, J., Golden, A., Medway, F., Myers, C. (1991, 
November). Assessment as Faculty Development. Presented at the FIPSE National 
Project Directors' Meeting. Washington, DC. 

Golden, A. (1991, October). Area Concentration Achievement Tests: History and 

Literature. Presentation to the Appalachian College Assessment Consortium. Johnson 
City, TN. 

Golden, A., Smith, M., Cone, A., and Kidda, M. (1991, June). The Area Concentration 
Achievement Test (ACAT): Different strategies for assessing the p^chology major. 
Panel presentation at the Sixth AAHE Conference on Assessment in Higher Education. 
San Francisco, CA. 

McGraw, K. (Moderator), Golden, A., Johnson, R., Matthews, J., and Smith M. (1991, 

March). Outcome evaluation in undergraduate psychology. Panel presentation at the 
meeting of the Southeastern Psychological Association. New Orleans, LA. 
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Golden, A. (1990, October). How to Participate in PACAT: Specific Options and 
Procedures. Present^ to the South Carolina Higher Education Assessment 
Consortium. Myrtle Beach, SC. 

Golden, A. (1990, October). PACAT: A Third Alternative for Assessing Majors. Presented 
to the South Carolina Higher Education Assessment Consortium. Myrtle Beach, SC. 

Cook, C. (Moderator), Amiran, M., Golden, A., and Wright, B. (1990, June). Why reinvent 
the wheel? The gentle art of "borrowing" instruments. Panel presentation at the Fifth 
AAHE Conference on Assessment in Higher Education. Washington, DC. 

Golden, A. and Squire, D. (1990, June). A national survey of requirements for the 

psychology major. Poster presented at the meeting of the American Psychological 
Society. Dallas, TX. 

Golden, A. (1990, May). Project for Area Concentration Achievement Testing. Presentation 
at the 3rd Annual Conference on the Assessment of Institutional Effectiveness. 

Atlanta, GA. 

Crain, M. and Golden A. (1990, April). An Examination of the Validity of the ACAT in 
Psychology. Paper presented at the meeting of the Southeastern Psychological 
Association. Atlanta, GA. 

Golden, A. (1990, April). The PACAT Model and its Application to the Content Domain of 
Political Science. Roundtable presentation at the meeting of the Midwestern Political 
Science Association. Chicago, IL. 

Golden, A. (1990, April). PACAT: Results of a 26 state survey of psychology curricula. 

Paper presented at the meeting of the Southeastern Psychological Association. Atlanta, 
GA. 

Golden, A. (1990, February). Getting started in assessment: General observations and the 
PACAT model. Presentation to the faculty of Johnson C. Smith University, Charlotte, 
NC. 

Golden, A. (1990, February). Getting started in assessment: General observations and the 
PACAT model. Presentation to the faculty of Jacksonville State University. 
Jacksonville, AL. 

Golden, A. (1990, February). PACAT Assessment of Major Objectives. Paper presented at 
the Charlotte Area Educational Consortium Assessment Conference. Charlotte, NC. 

Golden, A. (1989, June). Item banks for building instruments. Joint session with Roy 

Hardy, Director of the Educational Testing Service’s Atlanta office at the American 
Association for Higher Education Fourth National Conference in Higher Education. 
Atlanta, GA. 

Golden, A., Hart, M., and LeDuc, P. (1989, March). Project for Area Concentration 
Achievement Testing (PACAT): Report of the psychology curriculum survey and 
discussion of strategies for meeting assessment requirements. Paper presented at the 
meeting of the Southeastern Psychological Association. Washington, DC. 
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Golden, A. (1989, March). How to get involved with PACAT in the construction of 
cooperative assessment instruments. Paper presented at the meeting of the South 
Carolina Higher Education Assessment Network. Rock Hill, SC. 

Golden, A. (1989, February). PACAT: An alternative approach to the constmetion of 
nationally normed examinations. Presentation to the faculty and administration of 
Winthrop College, Rock Hill, SC. 

Golden, A. (1988, October). Test item banks: A cross between standardized tests & faculty 
designed tests. Paper presented at the FIPSE National Project Directors' Meeting, 
Washington, DC. 

Golden, A. (1988, July). ACAT: A third alternative for assessment. Presented to the faculty 
of the Department of Psychology, University of Alabama, Tuscaloosa, AL, July 1988. 



Workshops/Discussion Groups Conducted by PACAT Staff (1988-92) 

Charlotte Area Educational Consortium (Golden, A.) 

Jacksonville State University (Golden, A. and Squire, D.) 

Johnson C. Smith University (Golden, A.) 

Northwest Missouri State University (Golden, A. and Squire, D.) 
Regis University (Golden, A.) 

South Carolina Higher Education Assessment Network (Golden, A.) 
Winthrop University (Golden, A.) 
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APPENDIX VI; Results of the Assessment Planning Survey 
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S6 



YES NO 



Considering Assessment 966 324 

(Only the 966 departments responding affirmatively are included below.) 

YES % 

Status: 

Discussing 378 39.1 

Planning 256 26.5 

Implementing 254 26.3 

Already Implemented 206 21.3 

Reason: 

Accreditation 388 40.2 

Administrative Decision 384 39.8 

Departmental Decision 379 39.2 

State Mandate 170 17.6 

Governing Board Directive 123 12.7 

Methods: 

Nationally Normed Examination 529 54.8 

Classroom Examination 409 42.3 

Senior Seminar 388 40.2 

Senior Paper 305 31.6 

Exit Interview 257 26.6 

Locally Developed Instrument 233 24. 1 

Portfolio 119 12.3 

Oral Examination 88 9. 1 

Consortium Test Instrument 83 8.6 

Juried Presentation 71 7.3 

Professional Examination 45 4.7 
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APPENDIX VII: Sample Materials Developed by PACAT 
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PACAT 

PROJECT FOR AREA CONCENTRATION ACHIEVEMENT TESTING 



The Project for Area Concentration Achievement 
Testing (PACAT), funded by a grant from the U.S. 
Department of Education Fund for the Improvement of 
Postsecondary Education, is a national project which 
performs surveys of academic curricula by content area, 
collects examination items from the faculty of 
participating departments, and constructs, distributes, 
and scores the Area Concentration Achievement Test 
(ACAT) at nominal cost to participating departments. 

In order that they might better reflect academic 
diversity, versions of the ACAT for each discipline are 
constructed which conform as closely as possible to the 
different content area emphases identified by the 
surveys. PACAT is responding to a need within the 
academic community for models intended for outcomes 
assessment where the emphasis is on the evaluation of 
curricula. 



ORIGIN OF PACAT 

In 1983, Dr. Anthony Golden developed a model for 
major field assessment for a consortium of psychology 
departments in Tennessee. By 1987, the model had 
been extended to serve additional consortia in social 
work and political science. Since its introduction, the 
ACAT has gained widespread acceptance as a result of 
its ability to meet differing departmental needs, provide 
curriculum specific feedback and assistance with 
intemretation, and its cost-effectiveness. In August of 
1988, PACAT was formed with a grant from the U.S. 
Department of Education Fund for the Improvement of 
Postsecondary Education (FIPSE). To date, over 7,000 
ACATs have been administered at 61 public and private 
postsecondary institutions in 28 states. 

SURVEYS 

PACAT is surveying curricula and developing 
examinations in several academic disciplines. The 
surveys ascertain the content areas required by 
individual departments for a major. The results are 
used to isolate common patterns of curricular offerings 
and requirements, identified by content area rather than 
course. Survey results are distributed to those 
departments wnich request them. Since 1988, surveys 
have been sent to 12 ,d 00 academic departments with 
returns in excess of 5,600. All 50 states, the District of 
Columbia, Puerto Rico and the Virgin Islands have 
been included in the survey group. 

Disciplinary Areas Included in Surveys 



Art* 

Agriculture* 

Biology* 

Chemistry 
Communication* 
Criminal Justice§ 
Literature in English* 
Geology 



History§ 

Mathematics 

Physics 

Political Science* 
Psychology* 

Public Administration 
Social Work* 
Sociology 



* ACAT Currently available 
§ ACAT Preliminary version available 1992-3 



TEST POOLS 

Because the faculty who teach a discipline are best 
suited to determine what a student should learn from it, 
the results of the surveys are used to solicit examination 
items from faculty at participating institutions. 
Participants are asked to voluntarily submit objective 
items which reflect material that they consider to be of 
primary importance to their area of expertise and the 
cognitive level at which the student is expected to 
manipulate the material. Supplementary materials are 
available to aid in the construction of these items. The 
item pool developed in this manner, with contributions 
from a large number of faculty at different institutions, 
is assumed to be representative of the educational goals 
of the participating departments. 

PACAT also can accommodate essay or "open ended” 
itenis and will assist in the development of scoring 
guidelines. The use of these items requires that faculty 
continue their participation in the role of readers when 
the items are administered. 

Current Item Pool Size 



Agriculture 400 

Art 100 

Biology 350 

History 350 

Literature in English 295 

Political Science 1490 

Psychology 2750 

Social Work 850 



CONSTRUCTION OF THE ACAT 

The test item pools are used to construct examinations 
(Area Concentration Achievement Tests) which fit the 
content area profiles constructed from the survey data. 
ACATs then are provided to the participating 
departments for a nominal fee which includes test 
booklets, answer sheets, scoring, score reports and 
assistance with interpretation. A variety of score report 
formats are available. The examinations are revised 
using items which are field tested in earlier versions of 
the test. Parallel forms are used to increase the overall 
amount of material sampled and increase test security. 

EXAMPLES OF ACAT CONTENT AREAS 

Literature in English 

American Modem (1860s to present) 

American to 1865 
British Medieval Period 
British Renaissance 
British Romantic 
British Victorian 
Linguistics 

Restoration/ 1 8th Century /PreRomantic 
Shakespeare 
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Political Science 

International Politics/Comoarative Government 
Normative Empirical Theory 
Public Administration 

U.S. Government-Institutions, Policies, Processes 

Psychology 

Abnormal 

Animal Learning & Motivation 

Clinical /Counseling 

Development 

Experimental Design 

History & Systems 

Human Learning & Cognition 

Personality 

Physiological 

Sensation & Perception 

Social 

Statistics 

Social Work 

Human Behavior in the Social Environment 
Policy 

Social Work Practice 
Research Methods 

OTHER INFORMATION 

Demographic information in the following areas is 
collected at the time of administration and provided in 
summary format with the score reports. 

Age, Gender and Ethnicity/Content Areas 
Taken/English as 2nd Laneuage/First Generation 
Student Status/GPA Overall and in 
Major/Handicap Status/Plans for Graduate 
Study /Transfer Status 

SENSITIVITY OF THE ACAT 

Several investigations are either completed or in 
progress to determine the sensitivity of the ACAT to 
educational history. For the psychology ACAT, 
preliminary data indicate that students who are 
completing their first introductory level course in 
psychology at four different institutions produce overall 
scores between the 7th and 15th %'iles while those 
graduating with a major in psychology score at the 
50th %'ile and those who have entered a graduate 
program in psychology score at the 70th %*ile. These 
data suggest that the ACAT in psychology is sensitive 
to a range of proficiency levels. Si milar findings have 
been reported for the ACAT in sojcial work. Additional 
research suggests that the ACAT in psychology is 
sensitive to classrwm performance across several years 
of study in the major rather than just courses taken 
during the senior year. 



WAYS TO USE THE ACAT 

Plan 1 . PACAT will supply and score graduating 
senior exit exams at $3 in the first year of test 
availability. Tlie cost rises to $5 per examinee in the 
second year, $8 in the third year and a ceiling of $10 in 
the fourth year. The cost per examinee includes the use 
of the test oooklet, an answer form, scoring and nyiiling 
of scpre reports. Shipping costs are added at the time 
of shipment. 

Plan 2 allows for the use of a pretest/post-test design. 
Students are test^ when the major is declared and then 
again as graduating seniors. The pretest exams are 
provided at 50% of their regular cost. PACAT will 
score these exams and hold the scores rather than 
report! njg them to the institution. Post-test exams will 
be provided for the regular exit examination price. 

Both pre- and post-test scores will be reported for 
students taking both tests. 

Plans 3 (under^aduate) and 4 (graduate! provide and 
score the ACAT for experimental purposes at a cost of 
$3 per examinee under the conditions that an 
experimental protocol is submitted to and approved by 
PACAT and mat the ACAT is credited by name in all 
publications and presentations resulting from its use. 

Credits for Unused Booklets : Departments returning 
test booklets unused will receive a credit e<iual to $3 
less than cost per booklet. While the credit cannot be 
issued as a refund, it will be applied to the next order 
for test booklets. 



PACAT 
Box 4568 

Austin Peay State University 
Clarksville, TN 37044 

PHONE: (615)648-7451 
FAX: (615)648-7475 
BITNET: ANTHONY@APSU 



PACAT is funded by grant #P116B81711 from the fund for the 
Improvement of Postsecondary Education to Dr. Anthony Golden and 
Austin Peay State University. For FY90-91, Federal funding of 
$88,400 provides 70% of project operating costs with Austin Peay 
State University providing direct and indirect support of $38,144 
|30%). Austin Peay State University is an equal opportunity 
employer committed to the education of a non-racially identifiable 
student body. 

AP-961/12-90 
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PREPARING TEST ITEMS FOR PACAT 



In order for the Area Concentration Achievement Tests to reflect accurately the perceptions and 
priorities of participating departments, faculty are encouraged to submit test items for use on the 
test. When contemplating the subject matter of an item, please use the following guidelines. 

-Write in one or more of the areas listed in the PACAT curriculum to be used by your 
department. 

-Write in an area in which you contribute to the undergraduate curriculum in your 
department. 

-Write items that reflect your expectations of a graduating senior m^jor with respect 
to content and complexity. 

-Follow the format guidelines in the HOW TO SUBMIT TEST ITEMS TO PACAT 
document. 



MULTIPLE CHOICE ITEMS 

The Stem 

The most frequently used stem formats are the completion, incomplete sentence, and direct 
question types. Although used less often, a negative type stem also can be constructed. Consider 
the following direct question. 



What is the best description of the current concept of learning disabilities in 
general? 

Although it seems to ask a specific question, it uses the phrases "best description," "current 
concept" and "in general." Whenever a question asks for the "best" answer, it may be asking for 
a subjective interpretation and must limit the examinee to the listed alternatives. This question 
does not. The phrase "current concept" implies that one or more alternatives are correct older 
concepts which have been revised or rejected and should be used only if appropriate. The last 
problem is the phrase "in general." Is the intent of the question to ask for the best general 
statement or the best statement of the general category of disorders? Although the latter is the 
actual intent, it is not clear from the wording of the stem. 

The next example is of a negative stem which requires the identification of three or four correct 
alternatives to determine which is not correct. This type of item samples a somewhat greater 
breadth of information. The major objection to its use is that it also measures grammatical 
comprehension which is not necessarily relevant to its content. 

The operational portion of the learning disabilities definition states that a child with 
learning disabilities has a discrepancy between achievement and intellectual ability in 
one or more of seven areas. All of the following are areas in which a child could have a 
learning disability except 

This example contains information which can be used to answer other items. The following 
revision eliminates the unnecessary information while preserving the original intent of the item. 

Which of the following areas is not one in which a child could have a learning disability 
according to current Federal statutes? 
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The Alternatives 



Although writing the correct alternative takes little time, the distractors can be both difficult and 
frustrating. Good distractors should fall into one or more of the following categories. 

1 . the correct result of an incorrect procedure or application 

2. the correct answers in an incorrect order 

3. the result of a common error or misunderstanding 

4. a plausible response which is not correct 

Care should be taken that the distractors are in fact incorrect under all circumstances which may 
be explicitly or implicitly suggested by the stem. Care should also be taken that the correct 
alternative and the distractors are phrased in a similar fashion. Take, for example, the following 
test item. 



A. uneven physical growth patterns 

B. degree of academic difficulty 

C. discrepancy between potential and achievement 

D. degree of brain impairment 

The correct alternative, C, is longer than the others and is the only alternative which implies a 
comparison. If the examinee is aware only that a comparison is involved, the correct answer can 
be found. One solution is to rewrite the alternatives so that each contains a comparison. In some 
instances, both the stem and the alternatives can be reworded for brevity and clarity. If the 
comparison implied in each alternative above is made explicit in the stem, the alternatives can be 
shortened considerably. 

...a discrepancy between 

A. physical growth and age norms. 

B. academic difficulty and national standards. 

C. achievement and potential. 

D. neurological development and age norms. 

The incomplete sentence stem requires the selection of multiple answers contained within a single 
alternative. The appearance of the stem suggests that not only must the correct answers be 
selected, they must be in the correct order. Unfortunately, this type of item may not work as 
intended. 



Vital life support functions are controlled by the while muscle tone and fine motor 

control are controlled by the . 

A. pons, frontal lobes 

B. cerebellum, medulla 

C. frontal lobes, pons 

D. medulla, cerebellum 

Half of the alternatives can be eliminated by knowing that either pons or frontal lobes is an 
incorrect response or that either medulla or cerebellum is correct. Knowing only one of four 
pieces of information will reduce by half the number of alternatives. Once this decision has been 
made, it is necessary only to know one correct answer to select the correct alternative. As a 
result, an item intended to assess knowledge about two structures and the ability to identify them 
in the correct order actually may be measuring a variety of other response strategies. Increasing 
the number of items required may not eliminate this problem. 

The writer of test items for PACAT should avoid the use of "all of the above" type alternatives. 
In most instances, an examinee will seize upon this alternative whenever it appears that two or 
more alternatives might be correct. Just as in the case of the multiple answer alternatives, a 
correct response does not necessarily indicate knowledge about the full substance of the question. 
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Moreover, where only the occasional item has an "all of the above" or "none of the above" or "A 
and B above" type alternative, examinees tend to assume that it is included because it is correct. 

If all four distractors in a 5 alternative item are valid and well written, then the probability of a 
correct guess will be reduced compared to a 4 alternative item. Otherwise, it is better to write a 
good 4 alternative item than it is to add a poor distractor just to have 5. 

Other Types of Multiple Choice Items 

Questions which present information in one format and require the examinee to translate or 
interpret it are used often in the laboratory and social sciences. The presentation format can be 
any one which is different from that of the alternatives. It also is possible to present verbal 
information in the stem which must be matched with alternatives which are in another format. 




The mean of the distribution above is 

A. 4.000 

B. 4.296 

C. 4.333 

D. 4.370 

Alternative A is the mode rather than the mean. Alternatives B and D are the correct total 
divided by one more than and one less than the correct number of observations. The next 
example is of information presented in tabular format. Tables should be labeled completely 
unless labeling is being assessed. 

Given the following AIMOVA table: 



SOURCE 


df 


MS 


F 




Training (A) 


1 


1.15 


0.88 




Meal Pattern (B) 


1 


8.36 


6.45 


p<.05 


Illumination (C) 


1 


3.19 


2.45 




A X B 


1 


6.78 


5.21 


p<.05 


A X C 


1 


.75 


.57 




B X C 


1 


7.13 


5.48 


p<.05 


A X B X C 


1 


.09 


.06 




error 


72 


1.30 







A. Meal Pattern failed to account for a significant proportion of the 

variability. 

B. The effects of Meal Pattern were neither independent of Training 

nor of Illumination. 

C. Since Training was not significant, the interaction of Training with 

Meal Pattern can be Ignored. 

D. Since only Meal Pattern produced a significant main effect, the 

experiment failed. 
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A series of items can be based on a passage of text which describes a situation or problem. To 
measure the examinee’s ability to interpret and apply information, the situation described in 
the text should not be a standard example but rather should be constructed especially for the 
assessment instrument. The examinee should not be able to recognize the situation 
immediately and therefore select responses which already have been learned to match a 
particular situation or problem. The text passage should be complete in that it contains all of 
the information necessary to select the correct responses and should have sufficient details to 
permit the generation of numerous test items. 



A research study Is conducted using students who are required to take General Psychology. No 
attempt Is made to Identify Psychology majors among the participants. The subjects are told 
that they are participating In an experiment and are divided into four groups as follows. 

(A) Regular Instruction/Regular Text 

(B) Experimental Instruction/Regular Text 

(C) Regular Instruction/Experimental Text 

(D) Experimental Instruction/Experimental Text 

Regular instruction groups receive 3 lectures a week and must take notes, 5 minutes are set 
aside at the end of each class for discussion. Experimental Instruction involves discussion 
groups, frequent quizzes, and individual tutoring. The regular text is a standard textbook that 
has been In use for several years. The experimental text has been written for this experiment 
and self-test items, experiments that can be carried out by the students, and a number of other 
features. At the end of the term, all of the subjects take the same final examination and their 
scores are compared statistically. 

This experimental design is 

A. within groups 

B. 2 X 2 factorial 

C. 2 X 3 factorial 

D. Solomon 4-group 

The group which most closely approximates a control group for all of the 

experimental variables is 

A. Group A 

B. Group B 

C. Group C 

D. Group D 

A statistical comparison which looked only at textbook type would compare 

A. A + C vs. B + D 

B. A + Bvs. D + C 

C. A + D vs. B + C 

D. A vs. B * 4 * C * 4 * D 



ESSAY ITEMS 
General Information 

In the amount of time required to write a 20 minute essay, a student can answer 30-40 multiple 
choice items. The goal of an essay item should be to assess the examinee’s depth of 
knowledge and ability to manipulate and integrate that knowledge. As such, essay questions 
should extend well beyond the sample which can be obtained through a multiple choice format. 

Departments electing to include an essay component in their examinations must commit 
themselves to serving as readers for students at other institutions. The turn-around time 
typically will be 5 working days with faculty reading twice as many essay components as the 
number of ACATs their department administers. This insures two readers for each essay 
answer. Scoring forms will be provided along with each essay to be evaluated. 
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The Question 



Like multiple choice stems, essay questions should be complete and unambiguous. The 
question should delineate clearly the topic, issue or problem to be addressed without giving 
excessive cues as to the precise answer required. The question should be focused sufficiently 
to limit the range of answers to those that will best illustrate the desired skills and knowledge. 

It also is important to structure the question so that a complete answer can be written and 
reread in approximately 15 to 20 minutes. Essay components of the ACAT will contain three 
to four questions with an overall 65 minute time limit. 

Consider the following question. 

Describe the events leading to the War of 1812. 

For a test such as the ACAT, this question is far too broad. Author James A. Michener 
probably would begin an answer with the creation of the universe. For the examinee, no clear 
direction is given. The item could be rewritten as follows. 

Explain how the following factors led to the War of 1812. 

1 . The Napoleonic Wars 

2. U.S. desires for territorial expansion 

Avoid questions such as the following which simply require the student to list a series of 
points, a level of sophistication better measured by multiple choice items. 

List 3 factors which led to the War of 1812. 

Essay items for the ACAT should require the examinee to perform such tasks as arguing a case 
or point of view, discussing or explaining a relationship, constructing a novel solution to a 
problem, or reaching a documented conclusion. 

Answer Key 

Essay questions asked as part of the ACAT will be scored by two randomly assigned readers 
according to four criteria: originality; structure; content; overall impression. Guidelines as to 
content should be devised by the item's author in two categories. The first lists major points 
which the author feels must be included in a complete answer. The second lists additional 
points which should be included. These latter points are contributory to but not requisite for a 
complete answer. These points will be included with the question in the PACAT test item 
pool (see HOW TO SUBMIT TEST ITEMS TO PACAT). The lists do not have to be 
absolutely complete, readers will have sufficient latitude to accept good answers although they 
do not match the author's outline. 



ITEM IDENTIFICATION 

Each item, whether multiple choice or essay, should be identified by content area number 
according to the enclosed list. Authors also are asked to provide a short (1 to 3 word) 
description of the topic within the content area to which the item applies. An optional 
estimate of skill level required, determined according to the following taxonomy, can be added 
at the discretion of the author. 
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ESTIMATING THE DIFFICULTY OF TEST ITEMS 



Although the actual difficulty of an item is determined empirically after it is used, it is possible 
to estimate roughly the difficulty of a test item for the purpose of balanced test construction. 
This is done through a careful examination of the item and the cognitive skills that are r^uired 
to answer it correctly. Any of several taxonomic scales can be used for classifying test items. 

It is important to remember, however, that the determination of the skills required to answer a 
question must assume that only the minimum will be used. PACAT uses a hierarchy modified 
from Bloom (1981). The following is a summary of the four levels most frequently associated 
with multiple choice items. 

1. Knowledge 

This type of question requires the recognition and/or recall of basic facts and terms as 
well as specifics and details. It does not require manipulation, interpretation or 
analysis of the information. This is the least complex type of item. 

2. Comprehension 

This level goes beyond recall or recognition. It represents the most basic form of 
understanding, requiring the ability to restate or summarize information without 
altering its meaning. No abstraction or problem solving is required at this level. 

3. Application 

This level requires the application of general principles and techniques toward the 
solution of clearly stated problems. The problem is given or stat^ explicitly in the 
material and the examinee is required to apply the appropriate principle, techniques or 
theoretical approach toward a solution. The examinee is not required to produce new 
or innovative solutions. 

4. Analysis 

This level requires that the examinee be able to extract the salient features frorn a block 
of information and determine their implications. A higher level of abstraction is 
required at this level than at the previous level in that the individual must be able to 
respond to implicit as well as explicit features. 

The next two hierarchical levels typically are encountered only in essay items, although it is 
possible to construct multiple choice items that measure them. 

5. Synthesis 

This level has been reached when the examinee's response involves the assernbly of 
information, knowledge, and skills into a form which constitutes an explanation, plan 
or unique set of operations or constructs. The test item provides the necessary 
information for the response and specifies the task to be performed (explain, create a 
plan, construct a set of hypothetical relationships necessary to explain, etc.) but 
requires the examinee to assemble and integrate the necessary knowledge and extend it 
to the stated problem. 

6. Evaluation 

This level provides the examinee with information, cases or applications which must be 
evaluated either in terms of criteria which are independent of the content of the 
information or criteria which are determined by the content. The test item provides the 
necessary information and specifies the type of criterion to be applied. 

Classifying items on this hierarchy involves determining the highest level of cognitive 
achievement required. Each level contains some aspects of the levels below it, but also will 
require some additional manipulation of the information. 
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A Generalized Description of Important Steps 
for the Development of a 
Core Curriculum Assessment Program 



We have found through our visits to numerous campuses and workshops the following 
general guidelines for the implementation of assessment programs that provide feedback for 
future revisions and improvements. The program outlined here will eventually provide a 
continuous flow of information about the curriculum and what revisions should or should not 
be made in order to provide the maximum instruction available to the students enrolled. 



Step 1: You must ask yourself the question "What qualities should this person possess after 
completing this program?" 

These qualities should be viewed in terms of: 

- personal growth 

- development as scholars in preparation for upper division study 

This step should be addressed from a general perspective. We have found that the 
dialogue can be initiated simply by as&ng faculty to write a 1 paragraph description of 
their "ideal" student completing the general education program. These descriptions 
should be collated into a single statement which reflects the individual faculty opinions. 
The task can be addressed individually by departments participating in the core 
curriculum or overall. 

Step 2: What guidelines have been put in place to insure that these qualities are being 
developed? 

The guidelines should be expressed in terms of: 

- Course Objectives 

These are the specific objectives for the courses included in the general education core. 
Although the temptation is to address only the broad goals of general education, 
attention should also be paid to specific content. It is altogether possible that the 
content of the courses taken in general education is as critical to subsequent 
development as the broader skills such as critical thinking, writing, valuing, and so 
forth. The objectives ideally should be written in a manner that will permit assessment. 
That is, it should be possible to determine in a reasonably unambiguous manner 
whether or not the objective has been met. One of the best ways to begin this process 
is to begin with more specific objectives and, when faculty are comfortable with 
constructing measurable objectives at this level, proceed to the more general course 
goals. 



Example: ENGLISH I, Objective II. Given an editorial article selected from 
a national news magazine, students will be able to write an accurate summary 
of the article, clearly identifying the central theme, arguments, degree of factual 
support, and soundness of conclusion(s). 

Example: POLITICAL SCIENCE I, Objective 13. Students will be able to 
describe the nature of the economic relations between the United States, the 
European Common Market, Japan, and OPEC. 
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- Curricular Objectives 

Curricular objectives deal most directly with the issues raised in Step 1. For example, 
"Why should students be required to take the general education core?" Given that a 
general education core is a curriculum, then it must have specific overall goals. These 
are written in the same way as the course objectives above but deal with the overall 
goals of general education. Care should be taken not to simply extract goals from the 
course objectives but rather to craft a unique document that stands alone. 

Example: CRITICAL THINKING, Objective 3. Students will be able to write 
summaries of written materials dealing with topics in current domestic and 
international affairs, the sciences, and the arts, clearly identifying the central 
themes, arguments, degree of factual support, and soundness of conclusions. 

Example: CRITICAL THINKING, Objective 4. Students will demonstrate 
their ability to comprehend the specific dialogues of the academic disciplines by 
being able to write summaries of written materials in a manner consistent with 
the discipline from which the materials are drawn. 

- Structural Objectives 

The structural objectives should detail the integration of the course and curricular 
objectives. Given that the goals of the individual courses and of the overall core have 
been defined in a measurable fashion, the structural objectives detail the manner in 
which articulation is to be achieved and measured. The discussion at this point turns to 
the manner in which the various components of the core integrate with each other and 
contribute to the achievement of the curricular objectives of general education. 

Example: STRUCTURAL OBJECTIVE - END OF YEAR 1 , Objective 3. 
Prior to entry into the second level of the English and Science components of the 
core, students will have met Critical Thinking objectives 3 and 4. 

Step 3: Now that these guidelines are incorporated into the program, what methods can be 
employed to insure that these guidelines are working and can provide for further improvement 
in the future? 

The methods employed should: 

- measure the goals as defined by the faculty 

- be interpretable as they directly relate to these goals 

- contribute to faculty dialogue concerning the goals and possible revisions. 

This is perhaps the most difficult step to take. It is also the most creative and rewarding. 

Steps 1 and 2 have created a detailed "environment." Now, it is time to integrate assessment 
into the environment. The assessment techniques should be as varied as the curriculum and 
circumstances permit. In all cases, however, the techniques should be identifiable with the 
goals and objectives that they measure. The following examples are of different techniques 
which can be used for this purpose. 

Impact of differing entry skills and learning styles: Information on this dimension 
should be acquired at entry, either by accessing student records or through the 
administration of one or more of a variety of entry level assessment instrurnents 
designed to determine levels of self-esteem, skill, and learning style. This information 
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can be useful in the interpretation of other measurements as well as in the assessment of 
personal growth and scholarly development. 

Factual material acquired from core courses: The multiple choice format examination 
remains the fastest and most reliable method for measuring factual knowledge. The 
results of the test can be related directly back to the structure of the curriculum. 

Integrated assessment across the curriculum: Students should be r^uired to 
demonstrate retention of appropriate skills and knowledge from prior core curriculum 
courses. For example, class discussions, examinations (multiple choice, written, oral) 
and assigned papers (research, term, expository, creative) should assume that the 
student has met the goals of courses already taken. This level of integration also should 
be reflected in classroom lectures and syllabi. The existence of a clear and concise set 
of agreed upon objectives will greatly facilitate this process. Based upon the 
objectives, faculty also can differentiate performance more easily into the skills and 
knowledge which already have been acquired and those which are the goals of the 
present course. 

General skills assessment (rising junior and/or graduating senior): Either a nationally 
available instrument like the COMP, College Base, or Academic Profile, or an 
internally derived instrument should be used to assess the degree to which skills 
acquisition and retention have taken place. It should be kept in mind, however, that 
some instruments like the COMP are designed for graduating seniors and may not 
function well as a rising junior exam. Because one of the goals of a general education 
curriculum is to provide a foundation for subsequent study, the lasting impact of the 
core should be assessed in graduating seniors. The degree to which general education 
is meeting its goals is best assessed as students complete the core, before skills and 
knowledge are acquired and/or reinforced by a large number of other courses. 

Student surveys and/or interviews: A variety of excellent techniques are available for 
conducting satisfaction surveys and exit interviews for graduating seniors. These 
methods can and should be adapted for students exiting the general education program. 
When conducted toward the mid-point of a student's college career, these techniques 
can give students a strong sense of vestment in their own educational progress as well 
as providing valuable information about the general education curriculum. 

Continuing faculty discussion: Once begun, it is imperative that the faculty dialogue 
concerning general education be continued. Although a great deal of time and energy 
is spent assessing students, we often forget to collect dat^opinions/impressions from 
our faculty as well. If assessment is to be conducted, then clear-cut mechanisms should 
be in place by which to make those changes suggested by the results. Continuing 
faculty discussion and input appears to be an essential part of this process. 
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TEST CONSTRUCTION AND SCORING PROCEDURES 

FOR THE 

AREA CONCENTRATION ACHIEVEMENT TEST (ACAT) 



WARNING: INTERNAL WORKING DOCUMENT - DO NOT DISTRIBUTE, 
CITE, QUOTE, OR INCLUDE IN OTHER MATERIALS WITHOUT EXPRESS 
PERMISSION IN WRITING FROM THE COPYRIGHT HOLDER. 



COPYRIGHT 1990, 1992 by PACAT and Austin Peay State University. All rights reserved. 
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INTENT OF THE INSTRUMENT 

The Area Concentration Achievement Test (ACAT) is a multiple-choice assessment 
instrument primarily designed for the evaluation of departmental curricula against a national 
sample of similar departments. Although individual student scores can be obtained from the 
ACAT, they can be used only as general guides rather than as absolute indicators of individual 
performance. The ACAT is intended for use with graduating senior majors. 



TEST ITEMS 

Test items used on the ACAT can be either 4 or 5 alternative multiple-choice type. 
Both 4 and 5 alternative items can be mixed within a single version of the test. Faculty 
contributing potential test items are asked to write them to reflect their professional estimation 
of what a graduating senior should know and the manner in which they should be able to 
demonstrate the knowledge. That is, the cognitive skill level of the graduating senior should 
be reflected in the item as well as the content. The degree to which this charge is met is 
tempered by the format of the items (ie., multiple choice) and by the skill of the author. It 
should be noted, however, that increasing numbers of text passage reading items are being 
received as well as other types of item which appear to be capable of accessing higher order 
cognitive skills. 

Authors also are asked to classify their items according to a list of content areas 
developed by PACAT using national surveys and to use a 1 to 3 word phrase which denotes 
the general topic within the content area. These classifications are used for the determination 
of test construction specifications (see below). 



TEST CONSTRUCTION SPECIFICATIONS 

Test construction specifications are drawn initially from the curricula developed by 
PACAT through its national surveys. The surveys are constructed using 10 randomly selected 
college and university catalogs and comparing the descriptions of courses required for a major 
in the target discipline. Content areas appearing in at least 3 departments of the 10 are 
identified and included on the survey. Survey participants are asked to indicate, for each area, 
whether it is required, part of a list of options, or counted as an elective within their major. In 
addition, they are asked to indicate whether the content area is presented as a separate course, 
as one of several major topics within a course, or both. A write-in area is provided for the 
inclusion of content areas which do not appear on the instrument. Respondents are asked not 
to include content areas taught only in introductory level survey courses. Between October 
1988 and January 1991, surveys were sent to over 10,600 academic departments in the United 
States, Puerto Rico, and the Virgin Islands which are listed by the College Entrance 
Examination Board {Index of Majors, 1988) as offering 4 year baccalaureate degrees with a 
major in Agriculture, Art, Biology, Chemistry, Communication, Criminal Justice, Geology, 
History, Literature in English, Mathematics, Physics, Political Science, Psychology, Public 
Administration, Social Work, and Sociology. To date, approximately 5,800 completed 
surveys have been received and processed. 

The survey responses are tallied and each content area is coded according to the 
position it occupies within the structure of the individual major. Six categories of response are 
used with a seventh, "not offered," for content areas which either are unmarked or marked 
incorrectly. The "write-in" content areas also are grouped into categories where possible and 
tallied. 

The coded responses are subjected to an interactive modeling process to develop initial 
"proto-" curricula. Where possible, the model assumes that departments will share common 
content area cores of varying levels of complexity with additional content areas appearing in 
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required course clusters (students select one of several related courses) or electives. Most of 
the PACAT curricular models, therefore, are divided into a content area core and a list of 
optional content areas from which departments select a fixed number to be included. 

Required core groupings are tested in decreasing order of complexity and refined until 
there is minimal further change in the number of departments meeting the criterion for 
membership. Departments requiring a minimum of 80% of the content areas in a proposed 
core are assigned membership in the group. The matching departments are then extracted 
from the overall sample and a group of optional areas is designed which best describes the 
content area options offered by this group of departments. An appropriate number of optional 
content areas is determined by simple examination of the data. For the optional areas, a 
department meets the criterion if it requires 80% of the minimum number of optional content 
areas with more than half being taught as separate courses. An example of a curriculum model 
developed in this manner would be the Psychology "A" curriculum. This model has a core of 
eight required content areas with two options selected from a group of five. This modeling 
process is repeated with the remaining departments until as many of them have been matched 
with curricular groups as possible. 

Some disciplines (ie.. Political Science, Literature in English) do not lend themselves 
to a model of a core of required content areas with an additional group of discretely 
identifiable options. For these disciplines, models are developed in a similar fashion but with 
content areas weighted according to emphasis and grouped into more general clusters. 

In all cases, the curricula are intended to be used as the primary specifications for the 
construction of measurement instruments. The assumption is made that, while a curricular 
model may not fit perfectly any single department, it describes a structure in which students at 
different institutions will be at equal advantage with respect to the material covered. 

ACATs are designed to match the curricula developed in the manner described above. 
The actual test items used to measure the individual content areas are selected by a stratified 
random sampling process. Within each content area, a profile is constructed of the "topics" 
indicated by the item authors. The item pools then are sampled to reflect these topical 
emphases up to an overall maximum number of permissible items for the content area. In this 
manner, the ACAT attempts to represent an overall curriculum model common to a group of 
departments and to reflect the professional judgment of the faculty of participating departments 
as to the material content and cognitive expertise appropriate to a graduating senior. 



MULTIPLE PARALLEL FORMS 

Two parallel forms (Forms A and D) are constructed of each ACAT. The sampling 
procedure is repeated a second time, excluding those items which already have been selected. 
The result is two independent sets of items which reflect the same content emphases. 
Furthermore, by using both sets of items at the same time, a greater sample can be obtained of 
the performance level of a student cohort. This is particularly appropriate as the ACAT is 
primarily intended for curricular rather than individual student evaluation. To reduce the 
likelihood of cheating during the examination, each form is duplicated with the alternatives 
randomly ordered (Forms B and E). Test booklets are serially numbered with the forms in the 
order A/B/D/E so that no two adjacent examinees will have the same test booklet. Possible 
score differences between the forms are monitored carefully using the item analyses generated 
during the scoring process and by statistically comparing the individual content area 
performance of students on the two forms. 



TEST FORM VERIFICATION AND PROOFING 

Test items are edited upon receipt for general form, grammar, syntax, and agreement 
of stem and alternatives. The editing process is designed to reduce irrelevant grammatical 
distractors (such as superfluous colons) and reduce the impact of transitions from one writing 
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style to another. When necessary, authors are asked to clarify an item or others familiar with 
the field are asked to assist in revision. Once constructed, the test forms are verified 
individually to insure non-duplication of items and that items do not answer each other. The 
test forms are read and proofed by two members of the project staff for format, numbering, 
and appearance. 



CONTENT VALIDITY 

Departments, in consultation with PACAT staff, select the curriculum pattern that they 
wish to use. Prior to the use of the instrument, departments are urged to review and comment 
on the test contents. Where potential problems are identified with test items, faculty are asked 
to document the particular problem and either suggest corrections or draft a substitute item. 
Where potential problems are found with the material content of the instrument, faculty are 
asked to contribute items which they perceive to be more reflective of the state of the field. 
PACAT assumes that the large number of readers obtained in this manner is adequate to 
demonstrate the content validity of its instruments. 



SCORING PROCEDURES 
STAGE 1 

During the initial scoring stage, examinee responses are classified as correct, incorrect 
or blank. Correct responses are awarded 1 point while incorrect responses result in a .25 point 
penalty. The penalty has been necessitated by widely varying degrees of student cooperation 
and motivation with regard to this type of assessment process. Point-biserial correlation 
coefficients are constructed between individual item performance (correct/not correct) and total 



performance on the remaining items in the content area. 




STAGE 2 








During this stage, a 


series of item weights are developed which are based jointly upon 


the point-biserial correlations and the item difficulty index (percent correct). The catena tor 


these weights are described 


below. 






POINTS 


SIGN 


PROB 


% CORRECT 


0.0 


(+) 


<.01 


0- 9 


2.0 


(+) 


<.01 


10- 50 


1.5 


(+) 


<.01 


50- 90 


0.5 


(+) 


<.01 


90 - 100 


0.0 






0- 25 


1.0 






25- 90 


0.5 






90 - 100 


0.0 


(-) 


<.01 


0-100 



The item weights are reflective both of internal consistency and difficulty and ^e 
primarily intended to stabilize normative scores on tests during the early phases of their 
development. The criteria were developed during 1983-85 as a result of a number of 
simulations conducted using data collected using the original Psycholo^ ACAT. The average 
item weight for those items currently in use ranges between 1.2 and 1.6. 
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Correct responses are awarded the appropriate number of points for the item. A 
percent correct score is then calculated using the total number of points available for the 
content area. Items which are weighted with 0.0 points count neither in the score nor in the 
points available. No penalty is awarded for incorrect responses on these items. 

STAGE 3 

In the final scoring stage, the percent correct scores are translated into standard scores 
with a mean of 500 and a standard deviation of 100. Separate normative data are maintained 
for each content area as well as overall test performance, permitting each content area 
component to be treated as though it were a separate examination. Standard scores for the 
content areas and for overall performance also are calculated independently. This becomes 
increasingly important as the number of items on the instrument may differ by content area. 

By calculating the overall performance score independently, it can reflect the relative content 
emphasis of the test. The normative values consist of the cumulative percent correct scores for 
the preceding 4 year period. Each year, the earliest year's normative data are removed as 
another is added. This is done in order to accommodate gradual changes in performance 
resulting from curricular modification and improvement. In addition, individual normative 
groups consisting of all prior examinees in the discipline at a given institution are used to 
construct change scores for the individual department; These change scores are expressed only 
as .25 sd units. 



SCORE REPORTS 

Score reports are provided to participating departments following each of four annual 
testing cycles, with the final report being cumulative for the year. The types of information 
contained in the reports are selected by each department according to its needs. The 
information which is available consists of: 

- Overall departmental standard scores, percentiles, and cumulative sample sizes for 
each content area/content area cluster as well as for overall performance 

- Overall departmental average percent correct raw scores and standard deviations for 
each content area/content area cluster as well as for overall performance 

- Graphic presentation of distribution of departmental student performance scores in 
stanines compared to the overall group for the current testing year 

- Departmental change scores (described above) for each content area/content area 
cluster as well as for overall performance 

- Frequencies of examinee self-reports of classroom contact with content areas and 
point-bi serial correlations and probabilities for reportedly taking a course dealing with 
the content area and performance on the corresponding portion of the ACAT 

- Frequencies of examinee self-reported GPA overall and in the major (by range in 0.5 
point increments from 1.5 to 4.0) and product-moment correlations and probabilities 
for each type of GPA with the individual content areas and overall performance on 
the ACAT 

- Frequencies with which students report transferring credits in the major from other 
institutions 

- Frequencies with which students report planning to pursue graduate studies 

- Individual student standard scores (listed alphabetic^ly by last name or numerically 
by social security number) for each content area/content area cluster as well as for 
overall performance 

- Individual student scores in stanines (listed alphabetically by last name or numerically 
by social security number) for each content area/content area cluster as well as for 
overall performance 

Each section of the score report carries with it a brief explanation of the type of 
information that it contains. Warnings concerning the limitations of the scores and their 
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applications also are included. PACAT willingly provides consultation, in writing or by 
telephone, concerning the scores, score reports, and potential uses of the data. 



REVISIONS OF THE INSTRUMENT 

The ACATs periodically are revised with a typical cycle of 2 years depending upon the 
availability of new items. Items to be used for revisions typically are field tested as an 
unscored portion of the instrument for 1 year in order to determine whether or not they are 
appropriate for use. Appropriateness is determined in part by the weight assigned to the item. 
Items with weights between 1.0 and 2.0 are considered appropriate. Items are replaced if they 
fall into the 0.0 weighting category. Other items may be replaced in order to maintain the 
currency of the instrument. Items dealing with equivalent material may be exchanged across 
forms in order to balance the number of weighted points available. 



RESEARCH RESULTS 

To date, six research studies using the Psychology ACAT have been completed and 
either published or presented at professional meetings. A dissertation also has been conducted 
using the Social Work ACAT. All but one of these studies has been conducted independently 
of PACAT. The results of these studies conclude that the ACAT is at a difficulty level 
appropriate for graduating seniors (freshman/sophomores score at the 7th to 11th %'ile, 
graduating seniors at the 50th %'ile, first year graduate students at the 60th to 78th %'ile, and 
holders of the doctorate at the 92nd to 99th %'ile) and that it is sensitive to both curricular 
strengths and weaknesses as perceived independently by department faculty. A list of 
extramural research projects using the ACAT appears below. 

Dr. Elaine Ferraro 
Social Work Program 
Columbia College 
Columbia, SC 29303 
(803)786-3635 

(Dissertation using Social Work ACAT) 

Dr. Michael Smith 
Department of Psychology 
University of Tennessee 
Knoxville, TN 37996 
(615)974-6846 
(Validity Data - Psychology) 

Ms Michelle Crain 
Department of Psychology 
Memphis State University 
Memphis, TN 38152 
(Thesis - Validity Data - Psychology) 



Dr. William Chaplin 
Department of Psychology 
University of Alabama 
Tuscaloosa, AL 35487-0348 
(205)348-1926 
(Validity Data - Psychology) 

Dr. A1 Cone, Chair 
Department of Psychology 
Jamestown College - Box 6019 
Jamestown, ND 58401 
(701)252-3467 EXT. 2604 
(Research Data - Psychology) 
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PUBLICATIONS/OTHER MATERIALS 

Cone, A. (1988). Low tech/high touch criterion-based learning. Psychological Reports, 63, 
203-207. 

Cone A. (1990). Frequency of testing in criterion-based learning. Psychological Reports, 

’ 67, 396-398. 

Golden, A. (1991). PACAT: A national project in cooperative major field assessment. In 
R. McCormick (Ed.), The 1990 Montclair assessment conference: Strategies and 
prospects for the decade (pp. 80-83). Upper Montclair, NJ: Montclair State College. 

Golden, A. (1989). Project for Area Concentration Achievement Testing. Washington, DC: 
AASCU/ERIC Model Programs Inventory Project. (ERIC Document Reproduction 
Service No. ED 306 863). 

Golden, A. (1986). An approach to "In-House" test development for major field assessment, 
in K. McGuinness (Ed.), Legislative action & assessment: Reason & reality (pp. 
167-176). Washington, DC: American Association of State Colleges and Universities. 
(ERIC Document Reproduction Service No. ED 293 387). 

Golden, A., Smith, M., Cone, A., and Kidda, M. (1991, June). Vie Area Concentration 
Achievement Test (ACAT): Different strategies for assessing the p^chology major. 
Panel presentation at the Sixth AAHE Conference on Assessment in Higher Education. 
San Francisco, CA. Tapes available from Mobiltape Company, Inc., 25061 W. Ave. 
Stanford, Suite 70, Valencia CA 91355 (Tape ^91NCAHE-99). 

Golden, A. and Squire, D. (1991). The third alternative. Assessment Update, 3 (2), 10-11. 
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ACAT 

Score Report 

Sample Score Report 
LITERATURE IN ENGLISH 

Test Year: 1990-91 Cycle: 4 Report Prepared: lO-MAY-91 



STANDARD SCORES FOR THIS DEPARTMENT COMPARED TO THE OVERALL NORM GROUP 

Raw scores are converted to standard scores for the purpose of interpretation. Standard scores enable raw 
scores to be compared, both within areas of the same test and between different tests. The mean and 
standard deviations are arbitrarily set. The most common scale used is with a mean of 500 and a standard 
deviation of 100. The standard scores reported below use this scale. These overall standard scores ar^ 
based on a comparison of this year's scores with a A year cumulative norm group for each content area and 
for overall performance. 





STANDARD 


%'ILE 


NORM GROUP 


AREA TESTED 


SCORE 




SIZE 


American to 1865 


532 


62 


37 


American Modern (1860*s to pre 


575 


77 


37 


British Medieval Period 


538 


64 


37 


British Renaissance 


442 


29 


37 


British Romantic 


569 


75 


37 


British Victorian 


419 


22 


37 


Shakespeare 


589 


81 


37 


Linguistics 


455 


33 


37 


Restoration/18th Cent./PreRoma 


563 


73 


43 


•OVERALL SCORE : 


• ;476:',, 


41 ■• 


37 


NUMBER OF EXAMINEES AT THIS INSTITUTION: 




10 



RAW SCORE SUMMARY FOR THIS DEPARTMENT ONLY 

Listed below are your average percent correct scores for each content area and for overall performance. The 
standard deviations indicate the degree of variability in your students* raw scores. 



AREA TESTED 


MEAN 

% CORRECT 


STANDARD 

DEVIATION 


American to 1865 


56 


22 


American Modern (1860*s to pre 


53 


21 


British Medieval Period 


61 


28 


British Renaissance 


42 


18 


British Romantic 


58 


17 


British Victorian 


13 


13 


Shakespeare 


61 


17 


Linguistics 


37 


10 


Restoration/18th Cent./PreRoma 


70 


20 


OVERALL SCORE 




12 
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BEST COPY AVAILABLE 



Sample Score Report - LITERATURE IN ENGLISH - 1990-91 



page 2 



CHANGE SCORES COMPARING CURRENT SCORES TO THIS DEPARTMENT'S PRIOR PERFORMANCE 

The change scores listed below are calculated using only the performance of students in your department 
accumulated across all prior administrations. The change scores are reported in increments of .25 standard 
deviations to reflect changes of less than one full standard deviation. Interpretation of these scores 
should be made cautiously. Changes of less than .5 standard deviations may not be highly meaningf^ul. 
Negative values indicate declines in performance. Although based on the same performance as the standard 
scores reported above, the two types of score are not directly comparable. 



AREA TESTED 


CHANGE SCORE 


American to 1865 


-0.25 


American Modern (1860*s to pre 


0.75 


British Medieval Period 


0.25 


British Renaissance 


-1.25 


British Romantic 


0.75 


British Victorian 


0.00 


Shakespeare 


0.00 


Linguistics 


-0.75 


Restoration/18th Cent./PreRoma 


1.25 


OVERALL SCORE : : 


.0.75 


Prior examination years included in 


comparison; 1 


Prior examinees included in compari 


son: 7 



COMPARISON OF STANINE DISTRIBUTION FOR THIS DEPARTMENT AND OVERALL 
GROUP DISTRIBUTION FOR THIS TESTING YEAR 



For the following distributions, standardized overall scores have been converted into stanines. Stanines 
are used as general indicators of performance where large measurement errors are possible or where placing 
individual performance within percentile bands is more meaningful than using specific standard scores a^ 
percentiles. The upper percentile limits included in each stanine are as follows: (1) A.O; (2) n.u; taj 
23.0; (A) AO.O; (5) 60.0; (6) 77.0; (7) 89.0; (8) 96.0; (9) 100.0. 
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RELATIONSHIP OF CONTENT AREAS TAKEN BY EXAMINEES TO CONTENT AREA SCORES 

This table indicates the frequency with which your students reported taking one or more courses in each 
content area in your department; the percent that this number represents of your overall group; and the 
calculated relationship (point-biserial correlation) between studying the area in your department and 
performance on the corresponding component of the ACAT, Significant positive relationships suggest that 
taking one or more courses which cover this content area is associated with higher scores on the 
corresponding portion of this test. Significant negative relationships suggest the opposite. 



COURSE CONTENT 
AREA 


FREQUENCY 

TAKEN 


PERCENT 

TAKEN 


CORRELATION WITH 
CONTENT AREA SCORE 


British Medieval Period 


8 


80 


-0,816 


British Renaissance 


2 


20 


0,054 


Shakespeare 


10 


100 




Restoration/18th Cent./PreRoma 


6 


60 


0,402 


British Romantic 


4 


40 


0,313 


British Victorian 


0 


0 




American to 1865 (7-024) 


3 


30 


-0,292 


American Modern (1860's to pre 


8 


80 


0,346 


Linguistics 


10 


100 




* Indicates a statistically significant relationship. That is, a relationship 


of this strength is likely to happen by chance fewer than 5 times in 100. 



SELF-REPORTED CHARACTERISTICS OF EXAMINEE GROUP 

Examinees ore asked to provide several types of personal information at the time of administration. The 
following table summarizes some of this information. 



GPA 


OVERALL MAJOR 


OTHER INFORMATION 


1 1.5 - 2.0 

2.1 - 2.5 

2.6 - 3.0 

3.1 - 3.5 

3.6 - 4.0 


0 0 

6 4 

0 2 

0 0 

2 8 


Number of Transfer Students: 0 

Planning Graduate Studies: 10 


Female Examinees: 8 

Male Examinees: 2 


1 Note: Some examinees do not respond on all items, producing different totals. 
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RELATIONSHIP BETWEEN CONTENT AREA SCORES AND SELF-REPORTED CPA 

This table indicates the relationship (Pearson's r) between the examinees' self-reported GPA and performance 
on the ACAT. Significant positive relationships suggest that higher GPAs are associated with higher scores 
on the corresponding portion of this test. Significant negative relationships suggest the opposite. 



AREA TESTED 


CORRELATION WITH 
OVERALL GPA MAJOR GPA 


American to 1865 

American Modern (1860's to pre 

British Medieval Period 

British Renaissance 

British Romantic 

British Victorian 

Shakespeare 

Linguistics 

Restoration/18th Cent./PreRoma 


0.509 

-0.100 

-0.107 

0.366 

-0.687 

-0.262 

0.084 

0.533 

-0.557 


0.784 
0.460 
0.313 
0.417 
-0.978 * 
-0.331 
0.043 
0.406 
-0.247 


■ bVERAL L, SCORE -’’'-:;'’ ■ 




V : 0.406 ; 


* Indicates a statistically significant relationship. That 
is, a relationship of this strength is likely to happen by 
chance fewer than 5 times in 100. 
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INDIVIDUAL EXAMINEE STANDARD SCORES 
(LISTED ALPHABETICALLY BY LAST NAME) 

The scores which appear below should be interpreted with due caution. The ACAT is designed to evaluate 
overall performance of a student cohort. The meaningful ness of scores for individual examinees therefore is 
limi ted. 



EXAMINEE 


NAME 




SOCIAL 

SECURITY 


CONTENT AREA SCORES: 
12 3 4 


5 


6 


7 


8 


9 


TOTAL 

SCORE 


%MLE 


ALUCARD 


WILLIAM 


X 


111220004 


489 


401 


367 


368 


454 


414 


390 


516 


456 


385 


13 


ARNELL 


ARLETTE 




111220008 


673 


538 


536 


418 


558 


550 


485 


594 


581 


552 


69 


FRANCIS 


JANE 




111220003 


412 


414 


397 


547 


482 


373 


469 


408 


527 


413 


20 


JOHNS 


CAROLYN 


M 


111220012 


614 


501 


438 


487 


416 


441 


524 


491 


402 


514 


55 


JONES 


ANTHONY 


J 


111220001 


661 


588 


638 


726 


709 


550 


575 


547 


634 


704 


98 


SAMONER 


KELLY 


A 


111220007 


548 


662 


618 


726 


567 


427 


591 


558 


598 


500 


50 


STOKER 


BRAM 




111220002 


459 


501 


372 


517 


473 


427 


591 


558 


598 


500 


50 


TUSLE 


JAMES 




111220010 


708 


708 


590 


719 


707 


621 


473 


575 


754 


765 


99 


UNDERMAN 


PHILIP 


A 


111220006 


501 


488 


541 


537 


492 


455 


524 


413 


429 


461 


36 


UILAGREN 


PHYLLIS 


Z 


111220005 


644 


588 


392 


418 


425 


468 


357 


480 


393 


439 


28 



CONTENT AREAS TESTED FOR THIS GROUP (LISTED IN THE ORDER IN WHICH THEY APPEAR ABOVE): 

1 - American to 1865 

2 - American Modern (1860's to pre 

3 - British Medieval Period 

4 - British Renaissance 

5 - British Romantic 

6 - British Victorian 

7 - Shakespeare 

8 - Linguistics 

9 - Restoration/ 18th Cent ,/PreRoma 
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INDIVIDUAL EXAMINEE STANDARD SCORES 
(LISTED NUMERICALLY BY SOCIAL SECURITY NUMBER) 

The scores which appear below should be interpreted with due caution. The ACAT is designed to evaluate 
overall performance of a student cohort. The meaningfulness of scores for individual examinees therefore is 
limited. 



SOCIAL 

SECURITY 


CONTENT AREA SCORES: 
12 3 4 


5 


6 


7 


8 


9 


TOTAL 

SCORE 


%MLE 


111220001 


661 


588 


638 


726 


709 


550 


575 


547 


634 


704 


98 


111220002 


459 


501 


372 


517 


473 


427 


591 


558 


598 


500 


50 


111220003 


412 


414 


397 


547 


482 


373 


469 


408 


527 


413 


20 


111220004 


489 


401 


367 


368 


454 


414 


390 


516 


456 


385 


13 


111220005 


644 


588 


392 


418 


425 


468 


357 


480 


393 


439 


28 


111220006 


501 


488 


541 


537 


492 


455 


524 


413 


429 


461 


36 


111220007 


548 


662 


618 


726 


567 


427 


591 


558 


598 


500 


50 


111220008 


673 


538 


536 


418 


558 


550 


485 


594 


581 


552 


69 


111220010 


708 


708 


590 


719 


707 


621 


473 


575 


754 


765 


99 


111220012 


614 


501 


438 


487 


416 


441 


524 


491 


402 


514 


55 



CONTENT AREAS TESTED FOR THIS GROUP (LISTED IN THE ORDER IN WHICH THEY APPEAR ABOVE): 

1 - American to 1865 

2 - American Modern (1860*s to pre 

3 - British Medieval Period 

4 - British Renaissance 

5 - British Romantic 

6 - British Victorian 

7 - Shakespeare 

8 - Linguistics 

9 - Restoration/18th Cent./PreRoma 
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INDIVIDUAL PERFORMANCE IN STANINES 
(LISTED ALPHABETICALLY BY LAST NAME) 

The scores which appear below should be interpreted with due caution. The ACAT is designed to evaluate 
overall performance of a student cohort. The meaningful ness of scores for individual examinees therefore is 
limited. Stanines are used as general indicators of performance where large measurement errors are possible 
or where placing individual performance within percentile bands is more meaningful than using specific 
standard scores and percentiles. The upper percentile limits included in each stanine are as follows: (1) 
4.0; (2) 11.0; (3) 23.0; (4) 40.0; (5) 60.0; (6) 77.0; (7) 89.0; (8) 96.0; (9) 100.0. 



EXAMINEE 


NAME 




SOCIAL 

SECURITY 


CONTENT 
1 2 


AREA 

3 


SCORES: 

4 


5 


6 


7 


8 


ALUCARD 


WILLIAM 


X 


111220004 


5 


3 


2 


2 


4 


3 


3 


5 


ARNELL 


ARLETTE 




111220008 


8 


6 


6 


3 


6 


6 


5 


7 


FRANCIS 


JANE 




111220003 


3 


3 


3 


6 


5 


3 


4 


3 


JOHNS 


CAROLYN 


M 


111220012 


7 


5 


4 


5 


3 


4 


5 


5 


JONES 


ANTHONY 


J 


111220001 


8 


7 


8 


9 


9 


6 


6 


6 


SAMONER 


KELLY 


A 


111220007 


6 


8 


7 


9 


6 


4 


7 


6 


STOKER 


BRAM 




111220002 


4 


5 


2 


5 


4 


4 


7 


6 


TUSLE 


JAMES 




111220011 


9 


9 


7 


9 


9 


7 


5 


6 


UNDERMAN 


PHILIP 


A 


111220006 


5 


5 


6 


6 


5 


4 


5 


3 


WILAGREN 


PHYLLIS 


Z 


111220005 


8 


7 


3 


3 


4 


4 


2 


5 



9 



TOTAL 

SCORE 



4 

7 
6 

3 

8 
7 
7 
9 

4 
3 



3 

6 

3 
5 
9 
5 
5 
9 

4 
4 



CONTENT AREAS TESTED FOR THIS GROUP (LISTED IN THE ORDER IN WHICH THEY APPEAR ABOVE): 

1 - American to 1865 

2 - American Modern (1860's to pre 

3 - British Medieval Period 

4 - British Renaissance 

5 - British Romantic 

6 - British Victorian 

7 - Shakespeare 

8 • Linguistics 

9 - Restoration/ 18th Cent./PreRoma 
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INDIVIDUAL PERFORMANCE IN STANINES 
(LISTED NUMERICALLY BY SOCIAL SECURITY NUMBER) 

The scores which appear below should be interpreted with due caution. The ACAT is designed to evaluate 
overall performance of a student cohort. The meaningfulness of scores for individual examinees therefore is 
limited. Stanines are used as general indicators of performance where large measurement errors are possible 
or where placing individual performance within percentile bands is more meaningful than using specific 
standard scores and percentiles. The upper percentile limits included in each stanine are as follows: (1) 
4.0; (2) 11.0; (3) 23.0; (4) 40.0; (5) 60.0; (6) 77.0; (7) 89.0; (8) 96.0; (9) 100.0. 



SOCIAL 

SECURITY 


CONTENT 

1 


AREA 

2 


SCORES: 
3 4 


5 


6 


111220001 


8 


7 


8 


9 


9 


6 


111220002 


4 


5 


2 


5 


4 


4 


111220003 


3 


3 


3 


6 


5 


3 


111220004 


5 


3 


2 


2 


4 


3 


111220005 


8 


7 


3 


3 


4 


4 


111220006 


5 


5 


6 


6 


5 


4 


111220007 


6 


8 


7 


9 


6 


4 


111220008 


8 


6 


6 


3 


6 


6 


111220011 


9 


9 


7 


9 


9 


7 


111220012 


7 


5 


4 


5 


3 


4 



7 8 



6 6 
7 6 

4 3 
3 5 
2 5 

5 3 
7 6 
5 7 
5 6 
5 5 



9 



TOTAL 

SCORE 



8 

7 

6 

4 

3 

4 
7 
7 
9 
3 



9 

5 

3 

3 

4 

4 

5 

6 
9 
5 



CONTENT AREAS TESTED FOR THIS GROUP (LISTED IN THE ORDER IN WHICH THEY APPEAR ABOVE): 

1 - American to 1865 

2 - American Modern (1860*s to pre 

3 - British Medieval Period 

4 * British Renaissance 

5 - British Romantic 

6 - British Victorian 

7 • Shakespeare 

8 “ Linguistics 

9 - Restorat i on/18th Cent./PreRoma 
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Introduction 



This evaluation was undertaken at the request of the PACAT 
project director, Dr. Anthony Golden, to provide a consultative 
assessment and recommendations from an "outside" perspective. 
Subsequent statements toward those ends are based on the follow- 
ing: a reviev/ of the original FIPSE grant proposal, year 1 and 2 
progress reports and continuation proposals; three PACAT-based 
professional papers; a one-day site visit to Austin Peay Univer- 
sity, including direct interviev/s with PACAT staff members Dr. 
Golden and Denise Squire, and four institutional officials; 
telephone interviews with four project participants and consumers 
of PAC/Vr services; plus various other presentations and materials 
encountered over the past three years. 



PAC7^T Project Objectives 

As originally proposed in March, 1988, The PACAT project 
involved three primary goal activities. Progress toward each 
over the first two years will be summarized and evaluated in 
turn . 

Goal 1 : PACAT planned to conduct a two-phase survey of 500 

departments - based on a 25% expected return from 2000 mailouts - 
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in 24 disciplines in 14 states to determine curriculum require- 
ment patterns for majors. Multiple-choice test items suitable 
for assessing graduating seniors' knowledge base in those major 
programs v/ould then be elicited. 

Evaluation ; Although these v/ere considered ambitious objectives, 
the numerical criteria have been significantly exceeded. As of 
this date, project staff report that approximately 10,128 mail- 
outs have produced over 4,300 returns, from all 50 states, D.C. 
and two territories, in 13 disciplinary majors. This accomplish- 
ment is highly meritorious in and of itself, and provides an 
extraordinarily comprehensive information base for the project's 
subsequent assessment objectives. 

Goal 2 : PACAT proposed to utilize the major curriculum patterns 

and test items from Goal 1 to construct achievement tests (ACATs) 
appropriate for outcomes assessment of exiting seniors in those 
majors as an important indicator of the major programs' effec- 
tiveness. Such AC/vTs v/ere intended to provide distinct advantages 
for program evaluation purposes over the other two traditional 
testing options in that (1) ACATs were to offer more internal 
validity than nationally standardized commercial tests since the 
content would be more individualized to the curricular emphasis 
of each local program, and (2) ACATs also v>/ere to offer more 
external validity than locally developed tests since comparitive 
norms v/ould be developed among programs v/ith similar curricular 
patterns, and even national norms might be possible. Further- 
more, ACATs would analyze test items by difficulty level to help 
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determine students' cognitive skill level (based on Bloom's 
taxonomy) needed for each item. It was projected that by the end 
of project year 2 several thousand ACATs in 5 majors would have 
been constructed and disseminated. 

Evaluation : As of this date, 3 , 000 ACATs have been constructed 

in only ^ majors (psychology, social work, and political science) 
with two others (English literature and biology) well underway. 
VJhile these numbers fall well short of anticipated project goals, 
three mitigating considerations are evident. First, even these 
lower levels of test construction represent a very substantial 
accomplishment. Second, the numerical criteria v;ere probably too 
ambitious from the outset. V7hile those criteria were derived 
from empirical projections based on similar activities in 1983- 
88 within a smaller group of Tennessee colleges, the 1988-90 
expansion produced workload progressions that were more geometric 
than arithmetic. Thus, even higher work rates resulted in lower 
production rates. Last, the main reason for shortages in Goal 2 
projections were the overwhelmingly high demands for Goal 1 
activities. AC/vT production objectives were delayed due to the 
need to analyze and organize the prerequisite survey and item 
submissions on which the tests must be based. Since this is 
unquestionably a strong indication of the need for PACAT's 
processes and products, the actual Goal 2 achievement levels are 
not only acceptable, they're commendable! 

Goal 3 : PACAT proposed to provide a scoring and reporting 




service for ACAT consumers. Based on a graduated temporal scale 
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ranging from $0-$10 per test over four years of use, PACAT would 
machine score test ansv;er sheets and provide a very comprehensive 
array of descriptive statistical summaries of results, at both 
the program and individual student levels. Income and report 
projections for the three FIPSE-supported years anticipated 
significant ACAT participant revenues to begin in year 2, and 
increase dramatically with increased reporting in year three. At 
this rate, the PACAT project v/as expected to become self-support- 
ing in four years. 

E valuation ; 7vs with Goal 2, the quantitative criteria for Goal 3 
have not been met, for qualitatively important reasons. The 
"domino effect" of analyzing far more surveys than expected and 
organizing far more major curriculum patterns than expected and 
refining far more test items than expected and constructing far 
more tests than expected, has produced a much slower ACAT ad- 
ministration and revenue production cycle than expected. The 
PACAT staff had a choice: maintain high standards for all 

participants in Goals 1 and 2, or meet productivity quotas in 
Goals 2 and 3. Clearly, they have chosen the best and most 
conscientious alternative for their participants, but at the cost 
of fev;e.r end products and concommitantly lower income than 
originally planned. Ironically, PACAT ' s outstanding successes in 
Goals 1 and 2 have resulted in major problems in Goal 3, and thus 
potentially critical financial shortfalls for post-FIPSE con- 
tinuation, especially in 1991-93. 





Summary Evaluation of PACAT Activities 



Strengths and Assets 

- Clearly, PACAT activities have demonstrated a great need for 
ACAT products and services among a wide range of higher educa- 
tion programs throughout the country. 

- The PACAT products and services developed thus far - including 
surveys, computer programs, tests, results analyses, reports, 

etc. - are first rate; as good or better than the much more 
costly alternatives nationally prominent publishing companies 
provide . 

- The PACAT staff is very diligent, productive, creative, flexi- 
ble, and easy to v;ork with. Dr. Golden is frequently singled out 
for praise in all these areas by participants, colleagues, 
administrators and PACAT staffers alike. 

- The major curriculum patterns and multiple-choice item banks 
constitute very important resources for facilitating assessment 
in the respective disciplines. 

- The ACAT multiple-choice item bank is a widely used, standard 
component of most major assessments - particularly useful for 
external validity comparisons - and therefore likely to be in 
high demand for the foreseeable future. 
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- In iny opinion, ACAT revenue production over the long run will 
meet or exceed original projections for self-sufficiency or even 
profitability. 



Weaknesses and Problem Areas 

- Given the level of responses to PACAT surveys, the scope of the 
original project is unrealistic. Completion of ACATs for 24 
disciplines is v;ell out of reach, and ACAT production for even 
the thirteen majors presently at some stage of process will 
likely take at least two to three more years with present 
resources (i.e., four or five years total, rather than the 
expected three) . The revised test production goal of nine by 
1991 is much more feasible. 

- Present staff-time is stretched thin, and Dr. Golden's PACAT 
time demands are particularly unreasonable. Although equipment 
needs appear to be adequately met, space and facility needs are 
also not satisfactory. 

- Given the project's resource limitations, the funding discon- 
tinuation by FIP5E in 1991, and the very high demand for early- 
and mid-phase PACAT services by participants, the project can be 
expected to run low on funds before its goals are completed. 

- PACAT ' s "local support base" at /vustin Peay - and to some 
extent in Tennessee - could be stronger. Partially due to 
workload and perhaps partially due also to administrative style, 
the PACAT project has operated largely in isolation. By that I 
mean that peer colleagues and administrative superiors - while 
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generally very positive regarding PACAT ' s work - do not feel 
affiliated, involved or responsible for its continuation. While 
understandable, this lack of supporting relationships with other 
departments and offices could prove a real liability in the near 
future . 

- As an assessor, I feel the need to also point out the measure- 
ment limitations of the "objective" multiple-choice item bank 
which constitutes the bulk of PACAT ' s services. For either 
program evaluation or individual student assessment purposes, 
this type of item v/ill always be more superficial and have 
potentially greater internal validity problems than some other 
methods. Therefore, multiple-choice achievement testing at its 
best should only be used as one component in a multi-method 
approach to assessing majors (as PACAT technical materials 
acknowledge) . 

Recommendations 

The most critical need for the PACAT project is funding for 
continued opejration for one to two years beyond cessation of 
FIPSE support, or until the project can become self-sustaining. 
Toward that end PACAT project staff should begin planning for 
1991-93 funding during the 1990-91 project year, and consider the 
follov/ing strategies: 

1. Reduce the range of disciplines for which ACATs will 
be developed by 1993. Continued efforts to elicit new 
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disciplines for early phase activities, while desirable 
in the long run, create insurmountable cost and work 
loads in the short run. 

2. Concentrate ACAT cycle-completion activities on 
disciplines which offer the most promise for 
relatively quick revenue production, i.e., majors 
with the highest enrollments and/or most promise 
for high percentage department participation (e.g. 
psychology, biology, English literature, mass com- 
munication, social v;ork, political science, etc.) 

3. Increase PACAT prestige (and therefore credibility) by 
seeking endorsements, approvals, or other formal and 
informal legitimizations from sanctioning agencies and 
other authorities (e.g. SACS, THEC, FIPSE, disciplinary 
learned societies and professional associations, etc.) 

4. Increase PACAT utility and "user friendliness" 
by producing and disseminating a new "sales/PR" 
brochure. (The present promotional materials are 
very good, but may be too lengthy, technically so- 
phisticated, and "assessment wise" for the large 
majority of higher educators - especially those 
without direct training and experience in measure- 
ment theory, statistics, program evaluation and 
other vital aspects of higher edvication assessment - 
which means most potential PACAT participants) . 

The proposed brochure should be short, preferably 
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one to two pages back and front. It should emphasize 
both the accountability and program improvement 
potential of PACAT, stress the internal and external 
validity advantages, give examples of appropriate 
common uses for ACATs, cite specific state and 
accreditor mandates ACATs can serve, and include 
testimonials from recognized experts and satisfied 
consumers. (This information should be useful to 
current and prospective participants alike.) 

5. FIPSE offers $8,000 in "dissemination" grants for 
fourth year project activities, and these funds 
should be sought and utilized for presentations, 
publications and other similar activities to expand 
PACAT ' s participant and income base for the future. 
The PACAT staff's idea for a national conference 

on assessment of the major is an excellent one, 
potentially providing opportunities for partici- 
pant recruitment, revenue production, and broader 
benefits from the project in one package. Perhaps 
the FIPSE funds could be used as seed money for 
such a conference. 

6. Likev/ise, other grant support should be sought from 
state, federal and/or private foundation sources. 
Possible targets might include .Austin Peay, Tennessee 
and Federal Department of Education research and/or 
development grants. 
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7. Should only partial funding support be obtained for 
1991-93, consideration should be given to a "division 
of labor" collaborative effort with other assessment 
centers v/ho might be able to share production work 
and costs. Should acute funding shortages be faced, 
commercial publishers might "buy" PACAT products and 
procedures. This project has invested too much effort 
and produced too much good work to see it discontinued. 



In addition to continuation funding concerns, the following 
suggestions are also offered to improve PACAT operations, ser- 
vices and products: 

0. Based primarily on interview results, it is strongly 
recommended that concerted efforts be initiated or 
renewed to increase PACAT 's profile and participation 
at Austin Peay and among other Tennessee colleges 
and universities. This is emphasized not only for the 
potential revenue production, but primarily to con- 
stitute a core base of ACAT "models" to significantly 
broaden PACAT ' s impact as a program evaluation and 
effectiveness enhancement strategy. This effort should 
utilize Dr. Golden's considerable personal good will 
to establish PACAT ' s value on educational, profession- 
al and colleagial levels. (I got the definite impres- 
sion that key Austin Peay faculty, administrators, and 
other potential consumers and benefactors needed to 
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feel more affiliated with the PACAT project before 
they could become strong supporters. I know this 
suggestion exacerbates an avoidance-avoidance con- 
flict by further threatening Dr. Golden's teaching 
and PACAT operations priorities, but I would hate to 
see this project become "the best FIPSE project to 
ever fall short due to lack of PR", which I believe 
is a possibility without preemptive steps.) 

9. Along with some supplementing or revision of the 
participant "input" materials (Rec. #4) , I would 
like to see some modification of the ACAT results 
report. Consistent with the earlier cited "user 
friendliness", I'm concerned that recipients may 
well be overwhelmed with the breadth of descript- 
ive statistics provided, yet fail to appreciate 
tlieir potential depth of utility. Rather than - 
or in addition to - multiple tables of norm- 
referenced scores, an "interpretation guide" is needed 
expanding the current comments on the meaning 

of the results and placing them in a program 
evaluation context, in language that could be 
understood by assessment novices and veterans 
alike . 

10. It is also strongly recommended that PACAT 
continue to explore "item banking" and other 
technical assistance services for test options 
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other than multiple-choice questions (such as 
essay or short-answer items) . This would not 
only bring PACv\T ' s considerable expertise to 
bear on a potentially more valid assessment 
tool, but significantly increase PACAT ' s 
apparent value to participants as well. 

(Again, I know that additional time-intensive 
ideas are problematic, but from both a service 
and marketing viev;point I thought it warranted 
mentioning . ) 



In summary, my evaluation indicates that PACAT is a very 
important project providing valuable services to a very large 
constituency in a highly professional, productive, and account- 
able manner. What it lacks in physical and financial resources 
it makes up for in an unusually creative, conscientious and 
flexible staff, all spearheaded by the dedicated and enlightened 
leadership of Dr. Golden. With concerted efforts in the coming 
year to increase the utility of ACAT products, build a broader 
and stronger colleagial and administrative support base, and 
secure supplementary funding for 1991-93 activities, PACAT ' s 
considerable promise should be realized or exceeded. Speaking as 
a higher educator, an assessor, and a taxpayer, I'm extremely 
pleased with FIPSE's expenditure of funds on a a project as 
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AN EVALUATION OF THE PROGRAM FOR 
AREA CONCENTRATION ACIHEVEMENT TESTING 



INTRODUCTION 



During 1990, I was retained by the Program for Area Concentration Acliieve- 
ment Testing (PACAT) at Austin Peay State University (APSU) to act as an external 
evaluator. Prior to being asked to serve as an evaluator, I had several opportunities to 
meet with Dr. Anthony J. Golden, Project Director, and was familiar with the pro- 
gram. In addition, as Associate Director of the Center for Assessment Research and 
Development at the University of Tennessee, Knoxville (UTK) I am responsible for 
helping to prepare performance funding reports for the Tennessee Higher Education 
Commission, and I am well aware of the operation of performance funding in Tennes- 
see. 

On September 21, 1990 I had an opportunity to visit Austin Peay Stale Uni- 
versity to meet with the PACAT staff. I also had an opportunity to talk with Dr. Linda 
Rudolph (Assistant Vice President for Planning and Institutional Effectiveness), Dr. 
John Butler (Vice President for Academic Affairs), Dr. Carlton Stedman (Dean of the 
College of Education), and Dr. Stuart Bonnington (Chair of the Psychology Depart- 
ment) about the current role of the PACAT program at Austin Peay and its future. 

Given my background, I was asked to focus my evaluation of PACAT on its 
role in Tennessee and APSU, as well as its future after FIPSE support is no longer 
available. This evaluation does not consider questions related to the psychometric 
properties of the Area Concentration Achievement Tests or on PACAT's role outside of 
Tennessee except in the most general way. Having read Dr. Reid Johnson's review of 
PACAT's operations, and being in agreement with his conclusions and recommenda- 
tions, I have chosen not to repeat the points Dr. Johnson made. To better understand 
my comments regarding the role of the PACAT in Tennessee and at APSU, however, it 
is helpful to briefly review the history and purpose of PACAT. 
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BACKGROUND AND PURPOSE 

Tlie Program for Area Concenlralioii Achievement Testing began in 1983 as a 
consortium of Psychology departments. Headed by Dr. Anthony Golden, the goal of 
the consortium was to develop an outcomes measure in Psychology. By 1988, the 
consortium had expanded to include departments in Social Work and Political Science. 

Having received support from the Fund for the Improvement of Postsecondary 
Education (FIPSE) in 1988, PACAT currently serves 37 academic departments at 27 
institutions in 11 states. Starting with instruments in Psychology, Social Work, and 
Political Science, PACAT has expanded to include the development of outcomes mea- 
sures for Literature, Art, Communication, Physical Education, and Biology. 

PACAT measures provide a bridge between nationally-administered exams and 
locally-developed tests. Nationally-administered tests provide users with the advan- 
tages of high-quality measures and comparative data on student and/or program per- 
formance. However, these advantages are achieved at the cost of what is frequently a 
poor match between test content and the curriculum of a given department. In addi- 
tion, reliance on nationally-developed tests may make it extremely difficult to identify 
differences in student performance that are the products of differences in ability (or 
other pre-college characteristics) and program effects. One very real danger of too 
great a reliance on the scores from these tests is that assessment with result in "teaching 
to the test," rather than improvement to aid the learner. 

Because locally-developed tests can be tailored to a program's curriculum, 
tliese tests represent a much better match between test content and program cliaracteris- 
tics. This match also makes it easier to partition test performance into that which is tlie 
product of students' educational experiences and tliat which is beyond the influence of 
higher education. Unfortunately, tliese gains are frequently achieved at the costs of 
questionable test validity and tlie absence of comparative data for program evaluation. 

PACAT overcomes the limitations of both nationally-standardized and locally- 
developed tests by: (1) identifying content areas within academic disciplines; (2) cre- 
ating item pools for the common content areas, and (3) using the item pools to create a 
range of content area measures within disciplines from which institutions may select. 
Consequently, PACAT provides the flexibility and adaptability of locally-developed 
outcomes measures, while providing opportunities for the validation of outcome mea- 
sures and for the use of comparative data about program performance at other institu- 
tions. 
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This advantage of the Area Concentration Achievement Testing program can 
be clearly seen in the results of a recent study at the University of Tennessee, Knoxville 
(UTK) comparing the PACAT Psychology exam with the Major Field Achievement 
Test (MFAT) in Psychology which was developed by the Educational Testing Service 
(ETS). In this study, Psychology faculty and students at UTK rated the PACAT 
Psychology exam much higher than the Major Field Achievement Test in terms of its 
coverage of the department's curriculum, while noting that item quality is quite good 
for both exams. 

A second important advantage of the PACAT exams is that the test- 
development process can provide a useful vehicle for faculty involvement in asses- 
sment. This faculty involvement is absolutely essential if assessment findings are to 
result in learner-centered improvements in postsecondary education. 

By involving faculty in the item-writing process, PACAT stimulates a depart- 
ment's faculty to examine their curriculum and to identify the goals, objectives, and 
content areas within the curriculum Even when faculty are not directly involved in 
item writing, PACAT can stimulate faculty involvement through the selection of area 
concentrations to be included in the exam. My own personal bias is that encouraging 
faculty to critically examine their curricula is one of the most important benefits of the 
assessment movement. If this self-examination does not take place, test scores will do 
little to improve a program's quality and effectiveness. 



THE ROLE OF PACAT IN TENNESSEE 

The primary function of PACAT in Tennessee is to provide measures of in- 
stitutional and program effectiveness for the Tennessee performance funding program, 
which allocates a portion of public funds for higher education on the basis of the results 
of a series of assessment activities. Standard II of the performance funding guidelines 
awards funds, in part, on the basis of institutional means on cooperative tests, such as 
the PACAT Psychology exam. Because the objective of Standard II is to compare 
institutions on the basis of a single score, all Psychology Departments in Tennessee 
must use the same test to evaluate their programs. 

Other programs, such as Social Work, Political Science, Art, and Biology, 
may elect to use the PACAT exams in their disciplines. For these programs, effective- 
ness is judged on the basis of score improvement from one administration of the exam 
to another. Because the performance funding guidelines do not allow institutions to 
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alter more than 20% of a test’s content from one administration to the next, there is 
little variation in the forms of the exam over time. 

The fact that the PACAT exams must conform to the performance funding 
guidelines severely limits their utility for program improvement in Tennessee. Because 
a single version of the exam in Psychology must be administered at all public institu- 
tions, the PACAT' s advantage of flexibility is lost. The fact that some institutions have 
used the exam to make program improvements is commendable, but other institutions 
have found the form of the exam required by the Tennessee Higher Education Commis- 
sion to have limited utility in guiding curriculum reform and improving the quality of 
the learning environment for students. 

The previously mentioned comparison of the PACAT and MFAT exams in 
Psychology indicates that, while the content of the PACAT is judged to be somewhat 
more appropriate than the content of the MFAT, and while scores on the PACAT have 
.slightly higher correlations with coursework than do scores on the MFAT, none of 
these correlations are are statistically significant. I have little doubt that if the 
Psychology department at UTK was given an opportunity to select from the various 
area concentrations in psychology that arc available, it would be able identify a test that 
is related to patterns of course-taking in psychology. I also believe that the scores from 
this "tailored" exam would provide information that would be used to improve student 
learning. 

Even in fields where a single exam is not prescribed for the entire state, the 
performance funding guidelines limit flexibility by requiring that scores be comparable 
across administrations. If a department is successful in making meaningful changes in 
its curriculum, it is only natural that the department would want to subsequently modify 
the exam to better reflect the new curriculum. This option is severly limited in Tennes- 
see. 

The fact that a single, unchanging form of the PACAT examinations must be 
used in Tennessee also limits the extent to which faculty can be motivated to use test 
results for program improvement. Given the nonsignificant correlations between the 
PACAT Psychology exam and coursework at UTK, it is not surprising that faculty 
have shown little interest in examining their courses. Instead the prevailing philosophy 
is to "get the money and run." In fact, to revise the curriculum simply to improve 
PACAT Psychology scores would constitute "teaching to the test," rather than a genu- 
ine attempt at program improvement. It is important to remember that these limitations 
in the usefulness of the PACAT exams are products of the Tennessee performance 
funding program, not the exams themselves . 
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For the future, I do hold some hope that the current deliberations by the Per- 
formance Funding Advisory Committee and a proposed evaluation of performance 
funding by an external agency will create a climate in which tests, such as those offered 
by PACAT, will be used to improve programs and enhance student learning in Tennes- 
see. The potential for encouraging meaningful program improvement is enormous. 
Dr. Golden may want to consider making a formal presentation to the Performance 
Funding Advisory Committee concerning ways in which the PACAT exams could be 
better used in Tennessee. 



HIE ROLE OF PACAT AT APSU 

After speaking to several administrators at Austin Peay, I have the feeling that 
the PACAT is one of the best kept secrets on that campus. Not surprisingly, the PA- 
CAT exam in Psychology is being used by the Psychology department to critically 
review its curriculum and to make meaningful program improvements since this is Dr. 
Golden's discipline. However, I did not detect a similar commitment on the part of 
other departments. Perhaps this is because other departments are simply not aware of 
the potential benefits of the areas concentration exams. If the problem is lack of 
awareness, the fact that Dr. Golden recently received the Hawkins' Award and was 
given an opportunity to describe the PACAT program to the faculty may help to ex- 
pand its use. 

If there is not a growing interest in the use of area concentration tests at APSU 
during this coming year, the PACAT staff may want to consider working with senior 
administrators, such as Dr. Linda Rubolph, in developing ways to encourage APSU 
faculty to become more involved in the program. One way in which administrators 
may be able to encourage the use of PACAT' s resources would be to link this program 
with APSU's participation in the Dcming Quality Improvement program that will begin 
there this year. This linkage offers the possibility of improving both the quality of 
academic programs at APSU and suggesting new ways of reporting and using PACAT 
test results. 



CONTINUATION OF PACAT 

Given the enormous potential of PACAT, I am very concerned about the fu- 
ture of this program after FIPSE funding is no longer available. APSU has been very 
supportive of the program in terms of meeting its physical requirements (e.g., comput- 



er|c 



115 



PACAT EVALUATION 6 



cr hardware, office space, etc.). However, operating funds at APSU are limited and 
the support that can be provided may not be sufficient to allow PACAT to continue at 
its present level or, more important, to expand. 

Dr. Golden is already making every effort to secure outside funding for the 
program. He has, for example, asked the Tennessee Higher Education Commission to 
help support the program. I must admit to being somewhat skeptical about THEC 
support. First, I am not sure that the Commission with be willing to provide funds for 
operating the program outside Tennessee. Second, I am concerned that without a sig- 
nificant change in the philosophy of performance funding, THEC support for PACAT 
will result in a less flexible approach to test development, and a dilution of the pro- 
gram's benefits. Overall, I would strongly recommend that PACAT, with the assis- 
tance of FIPSE, seek operating funds for the program either from private sources or 
from another federal agency. 



RECOMMENDATIONS 

Overall, my exposure to the PACAT program leads me to believe that this is 
one of the best methods of obtaining outcomes data concerning student achievement in 
the major. The program has real potential for making significant learner-centered im- 
provements in higher education However, I do think that the full potential of the pro- 
gram has not been realized in Tennessee or at Austin Peay State University. I am also 
concerned that without additional support form external agencies, the program will not 
be able to continue at its present level or expand once FIPSE' s financial support is no 
longer available. Based on my concerns, I would offer the following suggestions to the 
PACAT staff: 

• PACAT should aggressively seek to influence the direction of the next five- 
year performance funding program cycle. The structure of the present per- 
formance funding guidelines inhibit using the area concentration exams to 
their fullest potential, and absent specific recommendations from PACAT, I 
do not foresee changes in the standard on assessment in the major that would 
encourage the use of achievement test scores to improve student learning. 

Linkages should be established between PACAT and the Deming Quality 
Improvement Program being instituted at APSU. This could help ensure 
wider use of the PACAT on campus and could suggest new methods of 
reporting and/or utilizing test results that would have important applications 
beyond APSU. 
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• PACAT, with assistance from FIPSE, should actively seek funds from pri- 
vate and/or federal sources to ensure the continuation and expansion of the 
program. Reliance on the Tennessee Higher Education Commission as a 
source of operating funds should be minimized. 



SUBMITTED BY: 
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Review of the Area Concentration 
Achievement Testing Program 

A review of the Area Concentration Achievement Tests (ACATs) and 
associated scoring and reporting features is indicated in the 
application for project approval by the Fund for the Improvement of 
Postsecondary Education. In conducting this review, I had access to the 
application, to examples of the psychology and political science 
achievement tests, to the "PACAT Process Description” (an internal 
working document dated August 1 , 1990 ), to user information available 
from the Project for Area Concentration Achievement Testing (PACAT), and 
to three professional papers describing the project and its instruments. 
This review addresses the question of appropriateness of the general 
procedures followed. Detailed concerns and suggestions for other 
procedures and services that might be offered have been communicated to 
the project director. 

The rationale behind the ACATs 

The legislative need to monitor the learning of college students in 
their disciplines of major study is evidently one important impetus for 
the development of the ACATs. The creators of the PACAT argue that both 
locally developed and nationally standardized achievement tests would 
not be optimum for satisfying the need. Local tests can be self-serving 
and lack internal quality and the ability to make comparisons in 
performance with similarly situated colleges. Natxonally standardized 
achievement tests can measure content that does not match in either 
coverage or emphasis the local program and can be a force to change the 
curricula of the individual college to conform with that measured by the 
test . 

The ACATs have the potential to mitigate the shortcomings of both 
local and nationally standardized tests. By pooling resources, the 
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cooperating institutions can develop tests that reflect rather than 
determine the institutions' curricular emphases^ that permit comparisons 
among institutions^ and that exceed the quality expected if only a 
single institution was responsible for the tests' development. 

Thus, I find the rationale and general approach taken by PACAT to be 
reasonable and appropriate. 

Creating the item pool 

The items that make up the ACATs are contributed by participating 
faculty from several colleges. Commendably, faculty have available 
item-writing instructions prepared by PACAT staff that go beyond the 
mere mechanical requirements for submitting items. Nevertheless, the 
absence of any serious checking of item accuracy means that poorly 
conceived items can enter the pool. If such items generate poor item 
statistics once they appear in an operational form of the test, they can 
be eliminated. But poorly conceived items do not necessarily have 
associated with them poor item statistics. Although I was not asked to 
review the tests item by item, in part because I do not have the subject 
matter competence to do so, I did review items in an area of my 
expertise and found 10 of 40 items that I felt were flawed and should 
not have been included in the ACATs . Other items could have been 
improved, although their integrity was not compromised. When inclusion 
of items depends only on the nomination of a single faculty member, it 
is not surprising that less than meritorious items will find their way 
into the tests. 

Aside from the issue of quality control, the items that faculty write 
and submit may over-represent areas that are easiest to measure. Thus, 
although each item may properly be a measure of a content area within 
the discipline, the item pool as a collection may not be representative 
of the full range of content within the area or of the full range of 
skills in dealing with the content. Again, I noticed that in the one 
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area I reviewed (statistics) r some topics were heavily questioned and 
recall of terms were emphasized at the expense of questions measuring 
higher-level thinking skills. 

I would recommend that a small group of able faculty from each 
content area review the items in the pool from that area with respect to 
individual item accuracy^ balance of subtopics, and inclusion of an 
appropriate range of skills. New items should be submitted to such 
review before they appear on new forms of the tests. 

Putting the ACATs together 

Tests are presently assembled for an institution by putting together 
those content areas appearing as an integral part of the curriculum for 
the major in that institution. 

Each form of the ACATs has four versions. Two versions contain 
different, but similar test items, which make it possible for judgments 
about the level of proficiency in a content area to be based on twice as 
many items than otherwise. It is indeed the case that (to be concrete) 

25 students answering one set of questions and a different 25 students 
answering a second set of questions will be more informative about the 
strengths and weaknesses of the curriculum than would submitting all 50 
students to a single set of test questions. The other two versions 
consist of the same items used in the first two versions, but placed in 
a different order and witli options scrambled. Such permutations guard 
against position effects (e.g., questions at the beginning of a test may 
get more attention and effort) and copying of answers. I applaud the 
use of multiple versions of each test form. 

Each of the patterns of content areas that are represented by many 
institutions will be the basis of a form of the test. In this way, the 
majors in these institutions will be assessed on content to which they 
can reasonably be expected to have been exposed. The decision of having 
a finite number of different forms is appropriate and reasonable. 
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although a case could be made for letting each institution put together 
its own form or for having a single form covering all topics and having 
even more than four versions of such a form. 

Scoring the ACATs 

Currently, not all correctly-answered items are worth the same number 
of points. Items with bad statistics are not counted at all (receive 0 
points) ; items with poor statistics are given some points but not as 
many as items with good statistics. This scoring procedure can lead to 
the situation in which a student who answers more questions correctly 
than another student actually receives a lower score. Is the practice 
of differentially weighting items according to their statistics 
acceptable? 

My answer is **yes**, with some qualifications. Counting those items 
that do a better job of measuring achievement more than less efficient 
items is reasonable. The seeming anomaly of answering more items 
correctly but scoring less would appear to be little concern given 
PACAT's emphasis on curriculum evaluation over student evaluation. 
Deside, illustrations abound where overall merit depends on more than 
the sheer number of good deeds . 

It is possible that the scoring weights, since they are determined 
completely empirically, could lead to some subtopics having less or more 
emphasis than intended. Also more optimum procedures exist for 
weighting the items than following the algorithm currently used. 

In summary, I approve of the present scoring procedures. The project 
staff should be alert that the item collection for a content area, as 
weighted, represents well the content domain. The staff may wish to 
consider other models for determining the differential item weights. 
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Reporbinq results on the ACATs 

Score reports are available showing individual student performance as 
well as school performance on each area and overall for the discipline. 
Results are expressed in both raw and standard scores. The reports are 
relatively easy to interpret. 

No formal equating of different tests has been undertaken. Although 
the four versions of each test form are likely to be similar, some 
differences are bound to exist. Similarly, not all area tests will be 
exactly equivalent, so the aggregate score for the discipline for one 
institution may not be comparable to that for a different institution 
that employs different area tests in its assessment. 

It may be that misinterpretations due to lack of strict equivalence 
in test reporting units are slight. Nevertheless, my recommendation 
would be carry out the formal equating work. 

Overall statement 

The PACAT is based on a sound rationale. Understandably, in the 
project's early stages, the staff was handicapped by limited human and 
financial resources and by modest usage. As the enterprise expands, the 
test development, scoring, and reporting procedures should add those 
psychometric practices that will permit the assessment program to 
achieve not only the high standards of the educational measurement 
community but also the ambitious goals it has set for itself. 



Jason Millman 



Ithaca, New York 
September 1990 
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August 4, 1992 



Anthony J. Golden, Director 
PACAT 

P O Box 4568 

Clarksville, Tennessee 37044 
Dear Dr. Golden: 

Thanks for inviting us to add conmients which might be used 
for your report to FIPSE. The Department of Psychology at 
Winthrop University began using the Area Concentration 
Achievement Test in the Spring, 1990. We administer this 
test to graduating seniors each December and May. These 
test scores are used, along with a variety of data from 
other sources, as part of our departmental assessment 
program. 

This instrument, and the PACAT staff, have contributed 
significantly to our assessment attempts. We will continue 
to administer the ACAT and look forward to continuing 
contact with you. 

Sincerely yours. 




Rondeau G. Laffitte, Jr. 
Professor of Psychology 



Rock Hill, South Carolina 29733 
(803) 323-2117 
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June 9, 1992 

Dr. Anthony J. Golden, Director 
Project for Area Concentration 
Achievement Testing 
P.O. Box 4568 

Austin Peay State University 
Clarksville, TN 37044 

Dear Dr. Golden; 

I should like to comment upon my experiences with PACAT. We are engaged in 
two ongoing research projects and your testing service is essential to both. 

1) At the University of Alabama we require that all graduate 
students in psychology teach an Introduction to Psychology 
course to a small class. The graduate students, theniselyes, 
enroll in a how-to-teach course, meet as a class and individually 
with the instructor, have their teaching performance evaluated 
in various ways, and take responsibility for grading and 
evaluating their undergraduate students. We have described 
this program in the literature on two occasions and we are 
presently testing the hypothesis that teaching the course 
increases the graduate students general knowledge of psychology. 

We have used your testing service to evaluate students who have 
taken the course compared to those waiting lists control 
students who will teach next semester. 

2) Our most ambitious project relates to outcomes assessment 
for our undergraduate psychology majors. We have developed 
content profiles based on the twenty-one areas which you identified. 

PACAT scores will be the criterion for several of our hypotheses 
about our undergraduate curriculum offerings. 

We are appreciative of all of the service which you have made available to us. In 
particular, we have appreciated your personal attention to, and interest in, our research 
projects. Many thanks! 

Sincerely yours. 




C. R: 



Henry C. Rickard, Ph.D. 
Professor and Chairperson 
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Department of Psychology 
325 Humanities Building 
Martin, Tennessee 38238-5059 
(901)587-7530 



June 8, 1992 



Dr. Anthony J. Golden, Director 

Project for Area Concentration Achievement Testing 
P.O. Box 4568 

Austin Peay State University 
Clarksville, TN 37044 

Dear Dr. Golden: 

We have always found the service on the ACAT in 
Psychology to be very satisfactory. This instrument has 
been quite useful to our department in its assessment 
efforts . 



Sincerely 




Gary E. Brown, Ph.D. 
Chair 
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Thomas Edison State College 
101 West State Street 
Trenton, New Jersey 08608-1 176 



Offlce Of Test Development 
And Research 
(609) 984-1140 



February 18, 1992 



Dr. Anthony J. Golden 

Professor of Psychology 

Director, PACAT 

Austin Peay State University 

P.O. Box 4568 

Clarksville, TN 37044 

Dear Dr. Golden: 



Thank you for the survey inquiry in relation to Mathematics. Thomas 
Edison State College does not itself offer any instruction, so that most of 
the survey questions will have no appropriate basis for response. Our 
students are all nontradit ional , working on a variety of degrees. They do 
offer mathematics coursework in these contexts, and for some of the degrees 
Calculus (Differential and Integral), Linear Algebra, and Statistics are 
required components. But I found no reference in our coursebook to any of the 
other courses specifically named on your General Purpose Data Sheet II. 



I was pleased to learn of your continued progress. I heard you speak at 
the Montclair State assessment conference in spring 1990, and we lunched at 
the same table. I subsequently wrote to you, but your project is defined a 
little differently than the approach that this College will most comfortably 
adapt to. I myself thought that PACAT was a very sensible and exciting 

(".'7crd3 not often usi::d to des.'ribe the same thing). If you return to 
Moutclair this year I would like to catch up on it. 



Good wishes for continued success. 



Sincerely, 

Thomas F. Donlon, Ph.D. 

Director, Office of Test Development and Research 
cmb 

Enclosure 

1514X 



BEST COPY AVAILABLE 



University of Alaska Anchorage 



12 1 1 Provklcncc Drive 
Andiura«e. Alask;i 99508 



COLLEGE Ol- ARTS AND SCIENCES 

Department of Psycholony 

(907) 786-1711 



July 7, 1992 



Anthony Golden, Ph.D. 

Director 

Project for Area Concentration Achievement Testing 
P.O. Box 4568 
Clarksville, TN 37044 

Dear Dr. Golden, 

This letter is in response to your request for feedback 
on our experience with the PACAT. We have just 
finished our second year of participation and find the 
PACAT in Psychology to be a useful curriculum 
assessment aid. 

All of our majors are required to take an exit test as 
a graduation requirement. No specific score is 
required - we view the test as program development tool 
rather than an individual assessment instrument. 
Presently our students can either take the PACAT or the 
Psychology subtest of the GRE to meet the exit test 
requirement. The requirement went into effect 
beginning with the 1990-1991 university catalog, so 
each year a larger number of graduating seniors take 
the exit test. Last year 50% of the graduates were 
included in the PACAT sample. 

The principal benefit to us from participation in the 
PACAT program is that the test scores cause us to think 
about the quality of our program. The PACAT reports 
focus attention on the outcome of our curriculum in 
terms of students' knowledge about specific content 
areas. This has been helpful and resulted in 
thoughtful discussions and some program changes. The 
PACAT is flexible enough that we don't feel our 
curriculum is being driven by the test. The freedom to 
alter the PACAT profile effectively removes this 
pitfall of outcome testing. The exit test requirement 
also serves us by making a public commitment to program 
quality. We plan to keep using the PACAT. The 
relatively low cost makes it an especially attractive 
program development aid during financially difficult 
times. 

Let me also mention that we have enjoyed the good 
service we have received from the personnel at Austin 
Peay . 



A DIVISION OF THE UNIVERSITY OF ALASKA STAJEmDE SYSTEM OF HIGHER EDUCATION 



Let me suggest one small improvement. As I understand 
it, group statistics for an institution are computed on 
each batch of answer sheets submitted for scoring. 

For our purposes, we would like to have one summary 
report per academic year even though we submit answer 
sheets each semester. Presently we administer the test 
three times per year but hold the answer sheets until 
spring so we can get them all included in one report. 
This gives us the largest possible sample for 
comparison with other programs. Students who took the 
test early in the year must wait until summer to get 
their scores. (We supply each student with stanine 
scores for information purposes. ) So my suggestion is 
to allow institutions to request that a single complete 
report be issued each year. If this involves extra 
work, a charge for this service might be appropriate. 
Smaller batches of answer sheets could then be 
submitted as collected and a grand report issued once a 
year . 

We hope the PACAT program continues. 




Robert J. Madigan, Ph.D. 
Professor 



CO Richard Bruce 

Undergraduate Program 




dinator 



From: IN%”U56076@UICVM . BITNET" "Stephen D. Spangehl, Associate Director" 

To: "Anthony J. Golden (615) 648-7451" <anthony(§APSU . BITNET> 

CC: 

Subj : PAG AT 

Return-path : <U56076(auiCVM . BITNET> 

Received: from JNET-DAEMON by APSU.BITNET with PMDF#10313; Thu, 1 Oct 1992 
11:20 GST 

Received: From UICVM(MAILER) by APSU with Jnet id 6944 for ANTHONY(aAPSU ; Thu, 

1 

Oct 92 11:20 GST 

Received: by UICVM (Mailer R2.07) id 3733; Thu, 01 Oct 92 11:19:05 CDT 
Date: 1 October 1992 11:15:35 CDT 

From: "Stephen D. Spangehl, Associate Director" <U56076@UICVM . BITNET> 

Subject: PACAT 

To: "Anthony J. Golden (615) 648-7451" <anthony(§APSU , BITNET> 

Message- id : <03FA8BB900600B14(3APSU . BITNET> 

Comment: "North Central Association" 

Comment: "Chicago, Illinois" 

Comment: "(800) 621-7440" 

Thanks for send me the info on your ACAT tests; I'm very impressed by 
what appears to me to be the most useful means I yet seen to collect 
this sort of data on academic achievement in the major. I'm so 
impressed that I gave your name to Don Wright (at Arkansas State U) and 
Linda Mann (at the Council of Independent Colleges) -- I hope you don't 
mind. Please keep me in mind and let me know of future developments, or 
of particular campuses where the ACAT tests are being used as part of an 
overall assessment program. 

Thanks again. --Steve 
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